Cloud platforms:
- Ceph (distributed object store and file system)
- Openstack (controls large pools of compute, storage, and networking resources)
Desktop backups in the cloud (I used all of these to form my opinion):
- Carbonite (works great, but doesn’t do NAS)
- Livedrive (completely sucks, restore doesn’t work for bunches of small files and it keeps interfering with Outlook by locking PST files)
- Backblaze (this is the one I use, works great and fast, but doesn’t backup NAS, very large files or locked files a.k.a. your e-mails)
- SOS Online Backup (only tested shortly, way slower upload than CrashPlan)
- CrashPlan (upload could be faster, does locked files, does NAS, does unlimited versioning)
Getting Amazon S3 to work with AWS CLI from PHP
- Installing the AWS Command Line Interface
- python –version (2.6.3 or later)
- cd /tmp
- wget https://s3.amazonaws.com/aws-cli/awscli-bundle.zip
- unzip awscli-bundle.zip
- sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
- aws configure
- Default region name: eu-central-1 (for Frankfurt)
- Default output format: json
- /etc/sudoers: comment out “Defaults requiretty”
- /etc/sudoers: add: apache ALL = NOPASSWD: /usr/local/bin/aws
- Test from PHP: `sudo /usr/local/bin/aws s3 ls s3://s3-bucket-name/s3-path 2>&1`
Amazon S3 AWS CLI
- Determine size of bucket: aws s3api list-objects –bucket s3-bucket-name –output json –query ‘[sum(Contents[].Size), length(Contents[])]’
- Location of configuration: ~/.aws/config
- Sync: http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html (e.g. aws s3 sync /root/synctest s3://s3-bucket-name/synctest)
- Remove: http://docs.aws.amazon.com/cli/latest/reference/s3/rm.html (e.g. aws s3 rm –recursive s3://s3-bucket-name/testroot/)
Mounting Amazon S3 as a Linux file system (CentOS 7)
- Exploring S3 based filesystems S3FS and S3Backer
- Please note s3fs will not work with Frankfurt as that requires AWS4-HMAC-SHA256
- Prepare for s3fs-fuse / s3backer: yum -y install fuse fuse-devel gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap make svn automake zlib-devel expat-devel pkg-config
- s3fs-fuse (includes working installation instructions for the latest version)
- Instructions how to actually use s3fs (ignore the rest)
- s3backer
- Bitbucket S3QL
- RioFS
Private Cloud / Sync
Syncthing
- Syncthing (SyncTrayzor)
- Syncthing in Synology DSM (install the beta package from synocommunity, then use this page to get it to actually work)
- Yum install on CentOS (repositories)
- Folder with settings: ~/.config/syncthing (edit config.xml to set the GUI address)
- (CentOS 6:) Edit /etc/rc.d/init.d/syncthing to remove syncthing_ARGS (to make config.xml leading for the GUI address)
- (CentOS 7:) Edit vi /usr/lib/systemd/system/syncthing\@.service to remove GUI address setting (alternative)
- (CentOS 7:) Use to start: systemctl start syncthing@root
- (CentOS 7:) As long as syncthing restart fails, use a crontjob to start it once per minute
- Build syncthing for Synology
- Install development tools: ipkg install optware-devel (see this for wget-ssl problems)
- wget –no-check-certificate https://github.com/syncthing/syncthing/archive/master.zip
- ???
- Latest binary releases (use syncthing-linux-amd64 for 64 bit CentOS 6 & 7 and for Synology, the 32 bit version sucks)
Bittorent Sync
- Main page at labs.bittorrent.com
- Unofficial FAQ
- Install btsync on centos 7
- Linux Setup, modes of operation
- Steps to get it working on a Linux server:
- Download and install btsync into /usr/sbin/btsync
- Create init.d script btsync (CentOS init.d startup script for btsync)
- make /etc/bitTorrentSync.cfg (Linux Setup, modes of operation), use valid dirs e.g. /etc/btsync
- chkconfig -add btsync
- ntsysv to enable auto start (check with chkconfig –list)
- There is an annoying bug in the Android version ignoring the WIFI only setting. A partial workaround is here: http://forum.bittorrent.com/topic/21474-bug-btsync-for-android-117-uses-cellular-data/
- Android KitKat 4.4 might also need this: SDFix: KitKat Writable MicroSD
Update Bittorent Sync – Synology version
- Install/update without starting the service
- cd /volume1/@appstore/bittorrentsync/
- chown root:root ./bin/btsync
- chmod a+s ./bin/btsync
Update Bittorent Sync – CentOS 6.5 x64
- Download BitTorrent-Sync_glibc23_x64.tar.gz from http://www.bittorrent.com/sync/download (e.g. to /root/tmp)
- (/root/tmp) tar -xvf BitTorrent-Sync_glibc23_x64.tar.gz
- (/root/tmp) service btsync stop
- (/root/tmp) cp btsync /usr/sbin/btsync or cp btsync /usr/bin/btsync
- (/root/tmp) service btsync start