mlbackup

Und wie benutzt man mlbackup? Im Terminal steht Usage: mlbackup path/to/configfile. Wie muss so ein file aussehen?

Ein kommentiertes Sample Konfig File wird in /etc/maclemon/backup/demo.mlbackupconf.sample installiert. Dort sind alle vier benötigten Parameter und der eine optionale Parameter erklärt.

Weitere Informationen findet man in der README Datei die beim Download enthalten ist.
Gruß Pepi

Spechen de English? Please?

Guten abend!

Dear Weston!

Yes, I do speak english as well. Documentation in english language is included with the download. The sample config file is commented as well so you should get started quite easily. Of course you can contact me in english as well if you have further feedback.
Best regards
Pepi

Hallo,
ich habe mal eine kleine Frage, die zwar off-Topic ist. Vielleicht haben Sie aber doch Lust und Zeit sie mir freundlicherweise zu beantworten:

Was muss ich tun um rsync als universal-binary zu compilieren?

Folgendes schlägt fehl bei mir:
./configure FLAGS="-isysroot /Developer/SDKs/MacOSX10.5.sdk -arch ppc -arch i386" LDFLAGS="-Wl,-syslibroot /Developer/SDKs/MacOSX10.5.sdk -arch ppc -arch i386 -arch ppc -arch i386"

make

ODER

./configure

make CFLAGS="-isysroot /Developer/SDKs/MacOSX10.5.sdk -arch ppc -arch i386" LDFLAGS="-Wl,-syslibroot /Developer/SDKs/MacOSX10.5.sdk -arch ppc -arch i386 -arch ppc -arch i386"

Oder setzen Sie lipo ein? Wenn ja, wie erzeuge ich dann ein PPC-binary auf einem Intel-Mac? Die Dokumentation von Apple ist da für mich als Hobbyprogrammieren ein zu kompliziert.

Ich möchte rsync in meinem eigenen Backup-Tool einsetzen (Applescript-Studio-Projekt) muss aber dafür an diversen Stellen den Source-Code ändern um das Protokoll von rsync besser "Maschinenlesbar" zu machen.

Vielen Dank und viele Grüße
Jürgen Venne

lipo wird bei einem Universal build automatisch verwendet um die resultierenden Binaries zusammenzufügen. Ohne komplette Logs kann ich da aber so nicht helfen. (Bitte diese auch nicht hier in den Kommentaren posten, das sprengt doch deutlich den Rahmen.)

Von einer Veränderung des rsync Quellcodes würde ich für diese Anwendung definitiv abraten. Besser man schreibt sich einen passenden Parser der die Ausgaben filtert und das gewünschte extrahiert. Beispielsweise mit sed, awk, grep und cut.
Gruß Pepi

Hallo,

vielen Dank für Ihre Antwort. Ich habe es jetzt geschafft. Apple hat hier ein Beispielprojekt, dass ich noch ein wenig ändern musste damit es funktioniert.
http://developer.apple.com/opensource/buildingopensourceuniversal.html

Den Quellcode ändere ich nur an 3 Stellen, um eben mit grep, cut und co. besser klar zu kommen.

Danke und viele Grüße
Jürgen Venne

Really like your backup-tool. The only issue I have is this one: I created a textfile with the name personalexclusions with following content:

- /tlinder/Docs/move
- /tlinder/Downloads/
- /tlinder/Library/VirtualBox/

#EOF

and set mlbackup to take this file:

MLuseConfigExcludeFrom=1
MLconfigExcludeFrom="--exclude-from=/etc/maclemon/backup/personalexclusions"

But the folders are still considered and written to the backup. Thank you in advance for your feedback!

Hi Tobias!

Thanks for your feedback!

Personal or per-backup-set exclusions are not yet fully implemented, hence they do not work properly yet. I am still working on this and hope to have this feature ready for an upcoming release. Please send me your mlbackup-configuration and intended use of a per-backup-exclusion file via eMail. (You can find the proper address in the mlbackup README file, please do not use the comments here for this.)
I want to get a better understanding for user's needs on this feature.

Thanks
Pepi

Hello,
I want to start mlbackup via crontab, but it doesn't work. I have added
05 03 * * * * root /etc/maclemon/backup/mlbackup mlbackupconf to /etc/private. Starting /etc/maclemon/backup/mlbackup in the terminal manually works fine. What have I to do?
I have OS 10.4.11.

Thanks,

Peter

Sorry, using the right path helps :-o I had tested it manually with the right path and wrote the wrong one in the crontab. Now it runs fine - thank you!
Peter

Good to see it is working and it was just a typo.

You always need to use an absolute PATH with cron as it is using a reduced environment and does not respect the usual search path for commands. The only way you could use a relative path with the config file parameter is if it was relative to the mlbackup tool itself. (Whis is a location where you should not place config files.)

Don't forget to mail me some feedback with your experiences about mlbackup. I'm interested in real world user reports!
Best regards
Pepi

Great tool,

I've just got one question: if I want to back up the entire drive (the boot-drive - I want to back up maclemon, too ;-) ), should I just leave MLsourcePath empty? And does maclemon then ignore /Volumes as I expect it will?

Thanks,
Matias

Matias,

mlbackup does know to except the /Volumes folder, use "/" as source. On the other hand mlbackup is not intended to be used for full device or volume backups. I will not create bootable backups or something you can easily restore a boot-volume from. There are other tools that are intended and hence more suitable for this task. I personally recommend CarbonCopyCloner for this task.
Pepi

Hi Pepi,

thank you for mlbackup, it now works properly. First I used rsync to backup my data, but the meta data was not copied properly. Then I fount rsyncX. It worked good, but I had only two backup sets. Now I have up to 14 ...
The only problem I had was to mount the external hard disk. I run my backups at night, my Mac mini boots automatically. mlbackup is started via cronjob. My mac does not login a user while booting. Therfore, the external HD is not mounted. So I have do it via script and cronjob before starting mlbackup:


#! /bin/bash
sudo mkdir /Volumes/Backup && sudo chmod 777 /Volumes/Backup
sudo mount_hfs /dev/disk1s1 /Volumes/Backup


This script is named mount_backup.bin.

sudo requires the administrator password. At night, nobody is there to type it in ... So I had to modify the sudoers file (visudo sudoers, the third line is added):


# User privilege specification
root ALL=(ALL) ALL
%admin ALL=(ALL) ALL
pdl ALL=NOPASSWD:/Users/pdl/Documents/mount_backup.bin


This way, mlbackup works every day at 3:05 while I am sleeping :-) Perhaps this helps other users.

Thank you,

Peter

Hi Peter!

Thanks for the suggestion. I have to disagree with your method. One should not really use sudo in a script. And by the way, all your hassle is not necessary at all.

Just issue
sudo defaults write /Library/Preferences/SystemConfiguration/autodiskmount AutomountDisksWithoutUserLogin -bool true
(all on a single line) once as admin, and Mac OS X (non Server) will also automount disks without user GUI login. (Like Mac OS X Server always does.)

This saves you from a potential local privilege escalation issue if someone can modify your script due to not-properly set permissions. Also saves you from modifying the /etc/sudoers file.

Hi Pepi,

thank you for this advice. This hint was searching for ... I have edited /Library/Preferences/SystemConfiguration/autodiskmount now and the sudoers file back to the origin, the crontab won't start the mounting script any more. I think it works this way. Tomorrow I will see ...

Thank you,

Peter

Hi Pepi,
thanks again, this works perfectly!

Enjoy your weekend,

Peter

Kleiner Bug(?) – Wenn ich die Mailaddresse leer lasse (wie in demo.mlbackupconf.sample vorgeschlagen) erhalte ich folgende Fehlermeldung:
mail: You must specify direct recipients with -s, -c, or -b.

Das rsync mir bei manuellem Aufruf die Fehlermeldung
file has vanished: "/Volumes/Noyoucant/.Spotlight-V100/Store-V1/Stores/BC9FC36B-29ED-4583-99C7-E79DC87143BB/live.0.i.dexUpdates"
gibt, liegt wahrscheinlich an meiner defekten Dateistruktur? Ich hab in den globalexcludes zumindest keine Erwähnung von Spotlight gefunden.

Hi Pepi

Thanks for the script. I am just wondering whether it is simply backup or whether it can sync please? I want the files and folders on a file server to be mirrored to a mac pro. Is this possible? I need the script to ignore permissions and also to run hourly. If the server isn't mounted can it ask to mount the server please?

Thanks

Rich

P.S. I am a total beginner and trying to find my feet!

Hallo (piep)!

Bitte schick mir ein Email mit der Konfigurationsdatei die Du verwendest, und gib' an welche mlbackup und welche Mac OS X Version Du einsetzt. Attache bitte auch das entsprechende rsync.log und error.log, dann kann ich mir ansehen wie Dein Fehler zustande kommt.

Spotlight wird in den aktuelle globalexclusions ab Zeile 134 behandelt und ist dort entsprechend kommentiert.

Dein manueller Aufruf von rsync hat nichts mit mlbackup zu tun.
Gruß Pepi

Hi Richard!

mlbackup is, as the name suggests, for backup purposes. It is neither intended nor suitable for synchronisation of directories. There are other utilities to achieve this.

mlbackup can be automated by launchd, cron or other utilities that can call it on a given schedule. (Might even be iCal when used with an AppleScript wrapper, though I definitely do NOT suggest to do so.)

mlbackup can not work reliably with a network mounted sharepoint as it cannot receive any notification should the network connection to the sharepoint get lost somehow. This is not recommended use and definitely will lead to problems. You will also get in trouble with file permissions and metadata. So don't use mlbackup to backup to or from a shared network drive. If you need to remotely pull backups from another Mac you will have to use mlbackup (rsync) with ssh-transport (and host key authentication) to have a reliable setup.
Best regards
Pepi

Hello Pepi and thank you for all the work you have done on mlbackup. It makes things easier.

I am trying to backup a set of files to an external volume, and it is working fine on OS X Leopard Server 10.5.6 with Cronnix GUI for cron (makes life a little easier). But, mlbackup does not seem to be honoring the MLbackupCount=2 parameter.

I noticed that the other parameter right sides have "" marks around the value.

Is this required for the MLbackupCount integer?

Please advise.

Kind regards, and Happy New Year,
Rick Cogley

Hi Rick!

What exactly do you mean by "does not seem to be honoring the MLbackupCount parameter"? Is there anything in the logs that might be of help? You can send me your configuration file and the logs via email if you like. You can find the address to send to in the mlbackup README.txt.

There is no need to quote the MLbackupCount integer, but it will not hurt if you do so. Check if you have any spaces between the number and the equal sign, there should not be any. The other parameters are file system paths and Unix will likely choke if you have spaces in them, that is why they are required to be quoted.
Best regards
Pepi

Hi Pepi!

Well, even though I set the parameter to 2, it keeps making more than that. It had done 4 copies, as of this AM. I set it up, let two days or cycles pass, then decided to look at it "in a couple days". When I looked again after 4 cycles, there were not the most recent 2 as I expected but rather 4 copies.

I confirm I have no spaces between the equals sign and the integer 2.

Kind regards,
Rick Cogley

Hi Pepi,
installed and tested the recent mlmac on our Xserve, and it works fine, so far.
but one thing for my comprehension:
if my config e.g contains a 5 step rollout, how to reinitiate simply a new full backup after that?
your help is really appreciated

regards forker

Hi forker!

There is no need to initiate a full backup as each and every backup is a full backup on the destination drive. Only new or modified files are transferred from source to destination. Identical files are hardlinked on the destination. This gives you the benefit of quicker backup times, and saved space on the destination. But each backupfolder in itself is a complete full backup which you can safely treat as such.
Best regards
Pepi

Vielen Dank!

Ein sehr nützliche Sache.

Trying to unarchive mlbackup-r125-3.0.5.tar.gz
I get this error:

Unable to unarchive "mlbackup-r125-3.0.5.tar.gz" into "00-test".
(Error 1 - Operation not permitted.)

I have checked permissions, they look good.

Is the problem the file?

Download fixed!

broomdodger,
The file got cut of during download by the webserver due to an incorrect file size in den database. The error has been corrected and the download now works fine.

If you want to check the downloads integrity, here are the digests you should get.
SHA1 = de2d49674f7c5dc4c171fbe2dd17aa3261df7da0
MD5 = d1ff5258aa7dbf9124492c812f8a6e5d

Sorry for the inconvenience!
Best regards
Pepi

hey pepi,

hab eine möglichkeit gesucht, dass wenn ich meinen usb-stick anschließe, automatisch ein backup von ihm gemacht wird. bin dann auf den block von holger krupp gestoßen mit einem script, basierend auf deinem program.
jetzt habe ich folgendes problem: wie dort auch schon diskutiert, wird immer ein backup erstellt sobald sich in "volumes" etwas ändert, egal was sich ändert und somit wird irgendwann das letzte backup gelöscht und mit einem leeren ordner "überschrieben". du hast da geschrieben, dass es seit mlbackup-r125-3.0.5 nicht mehr sein dürfte. bei mir ist das aber noch so. kannst du mir da weiter helfen (auf dem blog antwortet irgendwie keiner, bzw. werden meine einträge nicht moderiert)?

außerdem wollte ich noch fragen, ob es ne möglichkeit bei deinem program gibt, dass nur die veränderten daten kopiert werden? ich hab einen 4 gb usb-stick und ich glaub das es ziemlich nervig wäre wenn der voll ist, das immer die komplätten 4 gb kopiert werden.

bin erst seit kurzem auf mac umgestiegen und daher noch nicht so fit. hoffe du kannst mir weiter helfen.

beste grüße
moe

Hallo moe!

mlbackup kopiert immer nur die, gegenüber dem letzten erfolreichen Backup, veränderten Daten.

Das Problem mit dem automatischen Backup liegt eigentlich bei launchd. Dieser startet mlbackup immer wenn sich in /Volumes etwas ändert. Jedes aktivierte Volume ist eine Änderung. Jedes deaktivierte Volume ebenso. Dies ist das dokumentierte Verhalten von launchd und dagegen kann ich nichts tun.

mlbackup stellt fest wenn die in der Konfiguration definierte Quelle nicht vorhanden ist und bricht das Backup wieder ab. Bei korrekter Konfiguration sollte also kein Problem daraus entstehen, daß launchd zu oft feuert.

Bitte sende mir die von Dir verwendete Konfiguration und die Log Dateien per Mail zu, dann kann ich mir das genauer ansehen.
Gruß Pepi

Hi Pepi,

einmal mehr: herzlichen Dank für Deine Arbeit an mlbackup! Ein großartiges Tool das ich in so ziemlich jeder Installation verwende.

Nachdem oben schon mal cron erwähnt wurde, würde ich gerne noch auf eine weitere (IMHO bessere) Möglichkeit des Aufrufs aufmerksam machen - den launchd. Hier exemplarisch eine .plist mit der täglich ein Backup angestossen werden kann:


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Debug</key>
  <true/>
  <key>Label</key>
  <string>at.maclemon.mlbackup.daily</string>
  <key>OnDemand</key>
  <true/>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/mlbackup</string>
    <string>/etc/maclemon/backup/databackup</string>
  </array>
  <key>RunAtLoad</key>
  <false/>
  <key>ServiceDescription</key>
  <string>Kommentar was genau gemacht wird</string>
  <key>StartCalendarInterval</key>
  <dict>
    <key>Hour</key>
    <integer>0</integer>
    <key>Minute</key>
    <integer>55</integer>
  </dict>
</dict>
</plist>

als Datei »at.maclemon.mlbackup.daily.plist« unter /Library/LaunchDaemons/ sichern und mit launchctl starten. Wer das nicht zu Fuß anpassen mag, dem sei ein Blick auf http://lingon.sourceforge.net empfohlen!

Gruß Stefan

Hi Pepi,

Thanks for a nice frontend for rsync.

In my tests I have found out that mlbackup (or rsync) cannot create hardlinks on destination if target is a sharepoint on NAS or even a Mac OS X computer.
I have tried both SMB and AFP protocol when connecting to my QNAP NAS and both are failing.
I guess it is because my NAS has an embedded Linux and it has ext3 as its native filesystem???

I tried also using another Mac OS X computer as destination via AFP, same thing. It does not create hardlinks, amount of data on destination equals to how many backup sets has been configured.

Is this because UNIX system cannot create hardlinks on a remote volume or it is some kind of permission issue?

Maybe you should emphasize this in mlbackup documentation and maybe on top of this site also.

It may be just the way rsync or file copying over sharepoints works.
Only possible ways (that I know) how to get files hardlinked is to use local disk as a target or rsync directly to a remote host (this requires that target also has a compatible rsync running).

Hi Gnarf!

That is correct, to create hardlinks rsync needs native access to the filesystem and the filesystem has to support hardlinks! (FAT32 does not, Ext3 does.)

It is NOT recommended to backup to a network share over AFP, SMB, NFS, FTP, etc. mlbackup has no way to be informed about problems with the connection. I absolutely do NOT recommend to do this. mlabckup is not intended to cope with this because of the aforementioned reasons. It just cannot work reliably.

If you want to pull backups from a remote host you can do that. Install the same version of mlbackup on both Macs and use SSH hostkey authentication! mlbackup will happily use that as an encrypted and securely authenticated transport for rsync. Just specify user@hostname:path/to/source as the Source parameter. (SSH connection nicknames etc. from your ~/.ssh/config are supported.) If possible turn OFF compression for that SSH connection, mlbackup will compress by itself doing a much better job than SSH as it knows a lot more about the data.

Thanks for the suggestion about the documentation. I will add that information to the next release!
Best regards
Pepi

Hi Pepi.

First of all, thank you for your great work.

I have a little problem though.
When leaving blank the email recipient field, I get the following message :
"mail: You must specify direct recipients with -s, -c, or -b."
and nothing else happen.
Is there any workaround for this, as I don't have any mail server on my mac ?

And the most confusing is that mlBackup works with some sets, but not with some others, always with blank recipient field.

Thanks in advance

Hi Michel Antoine!

Please send me the details to the email you find in the README. Include mlbackup version, system version and the config files in question. Without this information it is impossible for me to help. Also indicate which sets are working, and which seem to be not. Include Logs wherever possible!
Best regards
Pepi

Hi Pepi,

I've recently tested mlbackup and would like to implement it's use on some of my servers.
I would like all backups created by mlbackup to retain their original file permissions.
My source and target files are located on the same volume.
I know there had been a switch within rsync to achieve this, but is there a way I can apply these settings using mlbackup?

Any help would be greatly appreciated...thank you.

Kurosaki-san he!

These are the switches according to the man page:

mlbackup usese these swtiches at all times to preserve users, groups and privileges (Posix and ACLs) as well as extended attributes (metadata) whereever possible.

--perms --executability --acls --xattrs --owner --group

So all the privs should remain intact and end up correctly on your backup. Do they?
Best regards
Pepi

Hello,

First of all: Very cool work and many thanks for this contribution to the community!

As some individual services are triggered by launchd (/Users/nicholas/Library/Favorites/:Library:LaunchAgents & /Users/nicholas/Library/Favorites/:Library:LaunchDaemons) and I have plenty of QuickLook plugins installed (/Users/nicholas/Library/Favorites/:Library:QuickLook) I am wondering how could I backup resources outside my $HOME-directory?

My approach so far would be to add a variable to /usr/local/bin/mlbackup which points to an additional file carrying the above mentioned resources. Unfortunately, it would be an intervention in the core of maclemon and I would be compelled to adapt it every time, when I update to a new version of maclemon.

Therefore, is there another kind of "trick" to backup resources outside the $HOME-directory?

Any help would be greatly appreciated.

Cheers,

Nicholas

Nicolas,

mlbackup follows a simple "one single source" to backup philosophy. So if you need to backup data from any directory, just create a config file to backup that one. Just specify it's path as the source.The default given in the sample configuration is just there to show that you can use environment variables as well as paths to specify sources or destinations.

Another way to get a few arbitrary files included with the backup could be to specify them in a per-backup exclude file. Sounds confusing and probably is. I have never tested this and therefor will not recommend this as it is not a supported feature of mlbackup.
Just create a per-backup exclude file and specify a path to your special file prepended with a + sign as the first character of the line.

Disclaimer: I have not tested this, and I have no idea how mlbackup will behave, so proceed at your own risk. Of course you're welcome to give feedback if it did work, bit your dog or ate up all your cookies.
Best regards
Pepi

Hello Pepi,

Swift response, great!

The hack with a preceding + sign ahead the resource doesn't work. It sounds to good, to be true!

However, configuring a 2nd backup-set for resources outside $HOME is also feasible for me.

Thanks one more time for your effort to find a solution for my needs.

Cheers,

Nicholas