Using Phing to sync files with shared hosting

Posted February 27th, 2009 by Juozas

PhingPhing is a project build system. It’s wonderful tool to automate every-days task by writing build scenario – Phing can do everything what can be done with PHP or Linux console applications. Today I was trying to use it in shared hosting server, but as you will see, Phing can’t do a lot of it’s functionality without direct access to server (over shh for example).

PHP::Impact blog has very nice article about remote files synchronization. Given FileSyncTask library uses rsync, console utility, which is de-facto standard for syncing files. Everything sounds great, but things get worse, when you try to sync files in shared hosting environment, where only FTP is available.

Code in the hole some months ago posted solution which uses Net_FTP (FTP functionality wrapper) package to upload files to server. I decided to try given solution, so I modified it to download source code from SVN first and then upload them. My tests code:

<?xml version="1.0" ?>
<project name="project" basedir="." default="build">
 
<property file="build.properties" />
 
<fileset dir="${sync.projectdir}/" id="files.main">
	<include name="**/*" />
        <!-- netbeans project folder -->
	<exclude name="nbproject/**" />
</fileset>
 
<target name="build" depends="svnexport,deploy-application-files" />
 
<target name="deploy-application-files">
	<echo msg="Deploying application files" />
	<phingcall target="deploy">
	    <property name="deploy.fileset.refid" value="files.main"/>
	</phingcall>
</target>
 
<target name="deploy">
        <ftpdeploy
	    host="${ftp.host}" port="${ftp.port}"
	    username="${ftp.username}" password="${ftp.password}"
	    dir="${ftp.dir}">
	    <fileset refid="${deploy.fileset.refid}" />
	</ftpdeploy>
</target>
 
<target name="svnexport">
	<delete dir="${sync.projectdir}" />
 
	<svnexport
	    username="${svn.username}" password="${svn.password}"
	    nocache="true" force="true"
	    repositoryurl="${svn.repo}" todir="${sync.projectdir}"
        />
</target>
 
</project>

Project was successfully uploaded, but it took more than half an hour (I have university Internet which is very fast). Problem is, it doesn’t check for files changes – it just uploads every single file in fileset. I don’t blame anyone, FTP is not made for syncing files and uploading huge amount of tiny files is very slow. We need other solution.

I have come up with another idea, which works by:

  1. Get last-revision-id from production server
  2. Download modified files (difference from last-revision-id to HEAD) from SVN repository
  3. Upload modified files
  4. Change last-revision-id to current ID

It works in theory and can be really simply implemented with Phing, but it’s still not very good solution. There is too much trust for one file (last-revision-id). Unless you can make sure that nobody will touch any files inside server without using Phing build script, things can easily brake. That’s why I haven’t tried it.

Have you used anything like this? Currently I’m thinking about moving to different server, because these limits (no ssh) for bigger projects creates massive headaches. I have came up with some hacks, as written above, but they are too sensitive and just mimics actual synchronization.

Trackbacks/Pingbacks

  1. Starting with Zend_Search_Lucene | Juozas devBlog

Comments (5)

  1. Daniel Cousineau

    I had thought about working through FTP but personally would probably off myself if I was limited to FTP and using Phing.

    All my stuff was done using rsync and could not be done over FTP (then again I’m also running some remote SSH commands like chmod and clearing out cache folders).

  2. Juozas (author)

    I would love to use rsync, but currently I’m limited to FTP-only server and our management plans now don’t include movement to normal dedicated server :)

  3. Daniel Cousineau

    You know, you could probably keep to the current implementation or have multiple build targets (one being “upload changes” the other being “upload everything”) and for any scripting you have to do upload a PHP script and have phing ping it with a one time key then delete it… *shrug*.

    Still doesn’t help the problem of syncing :P

    (Or maybe use the above idea but zip/gzip up the changeset, upload the zip, upload an installer/extractor PHP script dynamically generated with a one-time key, trigger the script, the script deletes the zip/gzip when finished, and Phing deletes the installer script… that might help bandwidth… but is certainly more work)

  4. Giorgio Sironi

    I think a great solution is using svn checkout directly on the server. However, you’ll need a shell access to do this…

  5. David Winterbottom (CodeInTheHole)

    I share your pain on the upload time when using FTP. As you indicate though, it’s not easy to find a better solution and the brute force method works fine for me – I just have a cup of tea while waiting for the transfer to complete.

    I needed that solution of a freelance project where changing the hosting is not an option. However, for all my other projects, I use NearlyFreeSpeech.net which is pay-per-use hosting and offers SSH access so you can use rsync (again through phing) for deployment.

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">