2.18.2008

Using Subversion externals property for WordPress upgrades

I find that upgrading apps like WordPress, Drupal, Symfony and open source PHP apps is simple for less complicated environments, but once you start adding in things like new directories, custom themes/modules, source code control as well as separate development, test, and production systems, the upgrades start to get pretty hairy.

Take WordPress for example. A typical upgrade of WordPress involves copying the new WordPress files over your existing files. Then you have to copy back safe versions of things like wp-config.php, .htaccess (if you're using it), as well as any custom themes/modules from the wp-content/ directory. Not to mention any of your own directories that should exist alongside your wp-includes/ and wp-content/ directories. After that you can run the upgrade.php file.

These upgrade steps aren't terrible. They're quite a bit better then most open source apps out there but they still suffer from a few problems:
  1. If you have your code, including your WordPress install, in Subversion or another source code control system, you have to commit all the files that change with each WordPress version. There may be files added, deleted, etc. You'll have to keep combing thru "svn status" messages to figure out everything you need to do to get all the WordPress files into your repository. This can be painful. And take a long time.
  2. WordPress is specifically written so that you don't ever have to muck with the guts of it. You create themes and plugins for added functionality. So, since you're not maintaining the code that powers WordPress, do you really need all those deltas in your Subversion repository? I think not.
  3. What do you do with your own code in directories that sits alongside wp-includes/? What if it’s in a Subversion repo?

The WordPress site also has instructions for using Subversion with your site. Here, they advocate the use of “svn switch” to update your site. This is much more manageable and solves a few of the above problems. Most svn users can probably can get away with this method. But unfortunately not me.

I have additional directories on some of my sites that I need to add into my WordPress install. So I have to copy/move them into the WordPress dirs which gets tough. And then my “svn status” will get all wonky because my WordPress dir is under one repo and my code is under another. This was endlessly confusing for me.

So I found myself looking for a way to completely wall off my WordPress install from the rest of my files. I was reminded recently of the use of the Subversion externals property and my mind started buzzing with possibilities. With "externals," I can say:
Pull the stable WordPress code from WordPress.org and put it into this directory named /docs/wp/

Then my other directories, which are under my own local subversion repo can exist at /docs/dir1, /docs/dir2, etc. Of course, some Apache Alias magic is needed to make all this work.

Here's the way I set it up for my some of my projects. So far so good. This is a bit hairy to set up but subsequent upgrades are a breeze. I use this across development, testing, and production systems (how to get those environments to work with WordPress will be another entry)

First off, the previous Apache document root for domain1 was at /www/domain1/docs, so the WordPress files wound up like this:

/www/domain1/docs/wp-content/
/www/domain1/docs/wp-includes/

But I also have a lot of dirs that sit alongside of wordpress like this:

/www/domain1/docs/dir1
/www/domain1/docs/dir2

We're going to wind up changing that.

Create an subversion external property in /www/domain1/docs for WordPress
[code]
$$ cd /www/domain1/docs
$$ export SVN_EDITOR=vi ( or your editor of choice )
$$ svn propedit svn:externals .
[/code]
vi starts up and you can add the following line:

[code]wp http://svn.automattic.com/WordPress/tags/2.3.3[/code]

Save and exit
[code]$$ svn commit
$$ svn update [/code]

( This downloads the WordPress code from the above address into your wp/ directory. Now we are cooking. )
  • Now, /www/domain1/docs/wp is where all your WordPress code lives.
  • Copy wp-config.php to /www/domain1/docs/wp/
  • Copy .htaccess to /www/domain1/docs/wp/
  • Create a link from the stock wp-content/ dir to your personal wp-content dir like this

[code]$$ cd /www/domain1/docs/wp
$$ rm -Rf wp-content/ ( use -Rf with care please! )
$$ ln -s /www/domain1/docs/wp-content wp-content
[/code]
  • In Apache config, set document root for this domain to /www/domain1/docs/wp
  • Put wp-content/ dir and any other non-WordPress dirs/files into /www/domain1/docs
  • Create an alias for wp-content/ and any other non-WordPress dirs/files in Apache config

[code]
Alias /wp-content /www/domain1/docs/wp-content
Alias /dir1 /www/domain1/docs/dir1
Alias /dir2 /www/domain1/docs/dir2
[/code]
This looks like a lot of work, but it's really only a lot the first time around. Next time WordPress has an upgrade:
[code]
$$ cd /www/domain1/docs
$$ svn propedit svn:externals . ( change the tag to new version of WordPress )
$$ cd wp/
$$ rm wp-content (to remove the link)
$$ svn update (to update to new version of WordPress)
$$ rm -Rf wp-content/
$$ ln -s /www/domain1/docs/wp-content wp-content
[/code]

Everything after the propedit in this group can and should be scripted which will basically give you a 2 step process for upgrading WordPress, while keeping you wp-content/ dir under local source code control, as well as leaving room for any other directories or files your site might require.

This technique will probably also work with Symfony although I haven’t tried it yet.

2.17.2008

"SSL Error: certificate verify failed" for Amazon S3 using s3sync

Amazon has changed their certificates for SSL access to the S3 service.  If you are receiving this error and you are using s3sync, you can look here for new certs for SSL access to S3.

Long term though, this is a problem with my (and possibly your) backup solution.  Look here for more info.  Specifically this bit:
The certificate chain supporting Amazon S3 SSL is an implementation detail of the system that may change from time to time. A robust application should not depend on the Amazon S3 SSL certificate being signed by a particular certification authority. However, you can depend on the fact that we will only use reputable CAs that are widely supported by existing user-agents. The easiest way to select root CAs to bundle with your application is to simply import the set from a modern web browser with a large market share.

1.17.2008

You know you’ve made it when the spammers start using your name.

All of a sudden, I have a bunch of returned email in one of my inboxes. Looking thru it, it appears that some spammer has been sending email thru various open relays around the world and using random names at this domain as the email address of the sender.
At first I freaked out thinking they were coming from my server. I looked around the box and didn’t see any evidence of abuse. And I’m not in any server blacklists either. On a whim, I tried connecting to some of the sending servers specified in the emails and sure enough, I could send email out from a couple, so I don’t believe there’s anything fishy going on with my server. I’ve notified the appropriate people regarding the open relays.

Most normal people will never have heard of my site. It’s not famous or popular. So when they get an email from an address at this site, they should immediately know that it’s spam. It’s not like sending an email out from a site that a lot of people use like the dozen “service at paypal” emails I receive each week. I’m just a rinky-dink site about programming that earns a few dollars a month in advertising, not even enough to cover my costs.

But I can see the spike in traffic this is creating as people either open the spam email or look at the sender and want to know more. (question – exactly who are these people that have time to research every single spam item they receive? I think I want their job ). With traffic going up, bandwidth usage goes up, and eventually costs go up. Traffic going up because more people are reading my site is great. Traffic going up because people are ticked off at me is not so great.

I’m still perplexed as to how to get this spammer to stop using my domain name as the sender’s address.

12.13.2007

People work with WS-* web services in PHP? Why?

I'd love to give ws02 a fair shake because they have an open source business model. They have a web services framework for PHP which seems interesting from an academic standpoint. But I think WS-* web services are WAY too complicated when compared to REST.

Just look at this new product description from ws02:
The WSO2 IS enables LAMP and Java websites to provide strong authentication based on the new interoperable Microsoft CardSpace technology, which is built on the open standards Security Assertion Mark-up Language (SAML) and WS-Trust.

Your life is now 1 minute shorter after trying to read and fully understand that paragraph. Nevermind how much time would be spent trying to actually get this stuff to work.

I spent a lot of time in 2001-2002 working with Amazon's merchant program and the SOAP feeds required for putting client product on the Amazon site. I generally think that Amazon has a pretty good clue about how to do things technically and they seemed to make it as easy as possible while using SOAP. But it was still way too complicated for what we were trying to achieve (ie, send a list of available product to Amazon to sell ). And at the time, it was a nightmare in PHP. Now of course, we have the official PHP SOAP extension and some items in PEAR to work with too (does nusoap still exist?)

But I've run screaming from WS-anything since then, only getting caught in its claws a few times. It hasn't gotten any easier. It's gotten more difficult. And more pointless. I'm not alone in this thinking.

So I guess the ws02 folks are trying to solve the issue of authentication for web services. Hasn't web services authentication been solved already in a much easier way too?