- What's for dinner?
- Should I accept that friend request on Facebook for the friend of a friend of a friend that I knew 15 years ago?
- What's the best development and test environment layouts for PHP using Apache as a web server with Subversion for version control for multiple developers?
Some of you may be asking yourselves the same questions. The choice of dinner is a personal one. I won't go into that except to say that everyone loves a good burrito. Spicy! And you probably don't care about my take on Facebook etiquette since your friends list probably dwarfs mine.
But I do have some definite thoughts on the layout of development environments. And I find that there's a huge lack of information about this on the interweb, so here you go.
We use Linux, Apache, PHP, and subversion in our development environment and so these instructions will be biased towards these topics but I think you can apply this method using various other technologies.
I like to give each developer their own development web site and development database. I find it's easier for everyone to have their own individual sandbox to play in. We give them each their own domain using their initials, something like rzdev.domain.com for me and vbdev.domain.com for another developer, Vinny Bag-o-Donuts. We set the Apache directories up on the Linux dev box in a similar fashion:
This has a few benefits. If I need to show Vinny something with my site development, I can just send him the link to http://rzdev.domain.com/broken-page. I can make changes to code, even major infrastructure code and not break anything for the other developers. We do the same thing with the databases, prefacing them with our initials.
Now, since our dev boxes use Linux, we set up Samba for sharing on these web directories. This means that all the devs can edit files and use source code management on the Linux server itself or on their Windows machines (we use either Eclipse or Zend Studio and create projects on the shares, that's a whole different posting!).
This dev site layout is closely linked to the way we use Subversion for version control. When we start a new site or application, if we can split out the development evenly enough, we'll just have everyone work from the trunk version of the code, with each developer working on their own little section. Each developer puts the trunk in their Apache dir and we edit the Apache configs to reflect this:
The root of the dev sites typically look like this:
/var/www/rzdev.domain.com/trunk/docs ( your Apache document root )
/var/www/rzdev.domain.com/trunk/lib ( non-public PHP library code )
When we commit code changes in Subversion, we have a hook that updates our main development site here:
Again, that site can be seen on the web at http://dev.domain.com/ This way, we can do integration testing on our code to make sure our new code doesn't break code from someone else within the dev site.
Now, the important thing here is that the Quality Assurance (QA) and Testing people ( if you're lucky enough to have them ), don't use any of these previously mentioned sites for their testing. Why not? Well, because if they're doing a good job and are therefore sufficiently anal, they're going to complain when code is changing on the site they're looking at.
So we give them their own test site and database that's viewed on the web at http://test.domain.com/ and setup in apache at:
The developers will meet and create the list of files and database changes that get moved over to the test site. How the actual moving is done doesn't really matter. If you have the time and energy to set up some Ant or Phing tasks, that works great. But copying/rsyncing files and running some SQL on the test database works just as well. The most important part is that the developers meet to decide which part can go to test. Otherwise, you could have code going to test and eventually production that might not be fully vetted.
When QA finds bugs in test.domain.com, they can send them to the developers. The developers can instantly start working on fixing the bugs in their own dev space at rzdev.domain.com and not affect the other developers or the ongoing testing of the application. Pretty nice right?
Advantages of this approach
- Uses source code management
- Developers can unit test their own code
- Developers can do integration testing between each other's code
- Developer A typically doesn't destroy code or data that developer B is using
- Developers don't destroy code or data that QA/testing is looking at
- Developers can both edit files and use source code management in either Linux or Windows environment
- Very scalable. Adding new developers into the mix is as simple as adding their respective sub domains and databases (of course, this can also be viewed as a disadvantage, see below)
- Less bugs make it to production
- Lots of sysadmin overhead initially and with each subsequent domain added. You have to set up all those developer sites, rzdev, vbdev, etc. Same overhead when using branching within subversion. Plus, you have to setup all those databases and setup the config code to connect to the appropriate database for each developer domain.
- Lots of file space for all the sites and databases.
- Confusing for lone wolf and gunslinger developers who are used to overwriting production or each other's development code (too bad for them!)
So what do you think? How do you setup YOUR PHP development environment?
What's annoying however is the way linkedin keeps track of company names and what happens when a company name changes. For instance, let's say I work at a company called "Davisons." I put that in my linkedin profile. "Davisons" gets bought by "Johnson, Inc" and they change the name of the company to "Johnson Davisons." So any new people coming into the company know it as "Johnson Davisons," not just "Davisons." When they look for current employees on linkedin, they won't find all of them because some have the old company named listed while some will have the new company name listed. Not to mention how bad this problem gets when one of your previous employers changes names after you have left.
This problem is what I would call an issue with name history. This kind of name history problem is starting to creep into other areas of the web as well. I've already had the misfortune of trying to email someone I haven't emailed in a few years and reached a completely different person (ie, one person's email acct was shut down and another started with the same address). Pretty embarassing -- makes me think twice about starting emails with "Hey dirtbag! Long time no see!"
The solution for linkedin's name history problem is pretty simple. They should ask you if the company has gone by any other names and let you choose from possible alternates (or even free type them). It won't take long for their system to build up the necessary relationships between past and current company names. It could even be pretty slick and auto-update old companies in your profile to their present name if the name is now different from what you specified.
A search for employees of "Davisons" in this case would give you both Davisons and Johnson Davisons and whatever other company names are "linked in" to this company.
I've run into the Lone Wolf PHP Developer at several places I have done work. Sometimes, I've had to work side-by-side with the Lone Wolf. Other times, I've replaced the Lone Wolf who had moved on to different hunting grounds. Still other times I've had to hire people and had to choose between a Lone Wolf and several other candidates.
Just who is this Lone Wolf and why should we fear them so much? Here are some telltale signs of the Lone Wolf PHP Developer:
- The Lone Wolf doesn't understand how to work in a team of developers. They typically don't even understand what benefits that would create. They do all development on their own, listening to very little input from qualified sources.
- The Lone Wolf got to page 141 on Enter-A-Beginner-PHP-Book-Title-Here and no further. Objects? They've never heard of them. They must not need them.
- The Lone Wolf re-invents the wheel for every project and doesn't use standard tools and practices.
- The Lone Wolf eats their young. OK, I made that one up. Frankly, how could a Lone Wolf have young anyway?
- The Lone Wolf is perfectly satisfied with doing programming work on production servers and using FTP to deploy their code. It never occurs to them that they should strive to create development and test environments. It never occurs to them that deployment via FTP doesn't scale higher then one developer.
- The Lone Wolf doesn't know what the letters SCM, CVS, or SVN are, or how to use them in their daily work.
- The Lone Wolf never reads my blog or any other blogs on programming. The Lone Wolf may not know what a blog is.
- The Lone Wolf says crazy things like "MySQL can't do transactions" and somehow gets management to believe them.
- The Lone Wolf was initially adored by management because they launched a lot of code live during their short stay. Too bad all that code is buggy and completely un-maintainable moving forward. Management doesn't like that.
- The Lone Wolf whips up incredibly stupid and unnecessarily complex solutions like template systems in which the templates are stored in a database instead of the file system/memory/cache. They shun using tried and true templating methods like PHP files, XSLT, or at the very least, Smarty. (see also: reinventing the wheel)
- The Lone Wolf names variables after themselves that mean nothing to anyone else (ex, $lonewolfFlag )
- And worst of all… The Lone Wolf PHP Developer fails to realize that there are other developers out there in the business world, trying to earn a living just like they are. By failing to conform to development standards that have been proven and tested, they make everyone's job more difficult.
Now, I personally have run into this Lone Wolf scenario with other programming languages as well. But I think because it is so simple to work with PHP without much formal training that it lends itself to this problem much more readily then other languages. The blessing and curse of PHP is that it is the new VB 6.
If anecdotal evidence means anything, and sometimes it does... and at the risk of sounding like a teenage girl writing on a myspace page, the PHP job market is literally blasting off! I've never seen the market for PHP programmers as strong as it right now in the NYC/Philadelphia region.
While there are lots of PHP jobs in the usual suspects like small start-ups, there also seem to be a number of jobs at established companies and developer shops. I'm also seeing a definite split in PHP jobs where there are both entry level jobs as well as jobs for more experienced people with titles like "Senior PHP Developer" and "PHP Tech Lead." I can't remember seeing so many senior PHP positions. The salaries I'm seeing are also at an all time high. Many ads have salary ranges that extend well into 6 figures.
As far as requirements I have seen, there definitely seems to be a shift towards frameworks like Symfony and Zend Framework as well as items specifically mentioning IDEs like Eclipse and Zend Studio. However, I rarely see mention of source code management and unit testing. Hopefully those items were cut from the job ads due to space requirements. ;-)
If you are a PHP developer in the area and are highly experienced, get ready to reap the rewards for your hard work. If you are a hiring manager in the area, prepare to shell out more for top talent.
So what is everyone else seeing in their area with regards to PHP developer jobs?
Check them out.
Google App Engine lets you run your web applications on Google's infrastructure. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it's ready to serve your users.
You can serve your app using a free domain name on the
appspot.com domain, or use Google Apps to serve it from your own domain. You can share your application with the world, or limit access to members of your organization.
Without actually firing up the SDK yet, here are my initial thoughts:
- Do we trust Google? With our data? With our users? I love the idea of the Google App Engine (GAE), a scalable web app system where you only have to worry about coding your business logic and structuring your models. This is kind of like Amazon’s EC2 on steroids. No sysadmin stuff like with EC2, just coding. However I do not love that Google is already in the web app space themselves and is now marketing a web app space hosting platform. You have to trust them a little bit more then I’m comfortable with. They've already got your users and your data and your code. Don't they conceivably control your business at that point? Maybe I'm being paranoid here.Plus, there’s the lock-in factor. If I develop my app using MySQL as a backend, I know I could with only some minor pain change that backend to Oracle. It’s all SQL at some point. How would you swap out of using the Google DataStore API? It’s a proprietary system with no published standards.
- GAE is currently only in Python. No PHP love? I realize a lot of Google runs on Python and C++ and that’s what it is in their wheelhouse. But I still consider Python to be a fringe language. If GAE were opened up to support PHP and C#, it would blast off in popularity. Then again, a Python only crowd is a good beta test before the crush of PHP devs comes in.
- The DataStore API (ie, BigTable) is difficult to wrap my head around. After spending so many years carefully crafting db tables and relationships to get the most bang for my buck, I now have to throw out a lot of that hard-earned knowledge. I need to think in terms of objects (or columns) instead of rows.There are still relationships and keys to keep track of but not in the traditional ways of SQL. It seems like much more of your data fetching is done in code rather then in SQL. The shift in thinking reminds me of my own move to Object-Oriented Programming. I whined like a baby during that phase of my programming development (well, at least I did on the inside).
I've also tried /tags/2.5 and /branches/2.5 and nothing. No changes come across. Very weird. Anyone else getting this?
Symfony was looking for me to have a __toString() function in my model so it could create the drop down menu. So for instance, let's say you have tables like countries and states in your schema.xml. Country_id in the states table is a foreign key to the Countries table like so:
When you do:
symfony propel-init-admin backend states States
Symfony is going to create the edit and create actions of the States screens with a drop-down menu of countries that the state is a part of. The value of the country drop down is the country_id. the value needs to be set in a __toString() function in your Country propel model. The easiest thing is to just return a simple string name (if that's in your db model):
That should solve the problem. It's odd that this comes up. And I couldn't find anything on it when googling.
Drupal has released version 6 now. So I click to the Apress site and lo and behold, they're already working on Pro Drupal Development, 2nd edition which will feature updates for Drupal 6. If it is as good as the first book, it will be a must have for any Drupal developer.
Take WordPress for example. A typical upgrade of WordPress involves copying the new WordPress files over your existing files. Then you have to copy back safe versions of things like wp-config.php, .htaccess (if you're using it), as well as any custom themes/modules from the wp-content/ directory. Not to mention any of your own directories that should exist alongside your wp-includes/ and wp-content/ directories. After that you can run the upgrade.php file.
These upgrade steps aren't terrible. They're quite a bit better then most open source apps out there but they still suffer from a few problems:
- If you have your code, including your WordPress install, in Subversion or another source code control system, you have to commit all the files that change with each WordPress version. There may be files added, deleted, etc. You'll have to keep combing thru "svn status" messages to figure out everything you need to do to get all the WordPress files into your repository. This can be painful. And take a long time.
- WordPress is specifically written so that you don't ever have to muck with the guts of it. You create themes and plugins for added functionality. So, since you're not maintaining the code that powers WordPress, do you really need all those deltas in your Subversion repository? I think not.
- What do you do with your own code in directories that sits alongside wp-includes/? What if it’s in a Subversion repo?
The WordPress site also has instructions for using Subversion with your site. Here, they advocate the use of “svn switch” to update your site. This is much more manageable and solves a few of the above problems. Most svn users can probably can get away with this method. But unfortunately not me.
I have additional directories on some of my sites that I need to add into my WordPress install. So I have to copy/move them into the WordPress dirs which gets tough. And then my “svn status” will get all wonky because my WordPress dir is under one repo and my code is under another. This was endlessly confusing for me.
So I found myself looking for a way to completely wall off my WordPress install from the rest of my files. I was reminded recently of the use of the Subversion externals property and my mind started buzzing with possibilities. With "externals," I can say:
Pull the stable WordPress code from WordPress.org and put it into this directory named /docs/wp/
Then my other directories, which are under my own local subversion repo can exist at /docs/dir1, /docs/dir2, etc. Of course, some Apache Alias magic is needed to make all this work.
Here's the way I set it up for my some of my projects. So far so good. This is a bit hairy to set up but subsequent upgrades are a breeze. I use this across development, testing, and production systems (how to get those environments to work with WordPress will be another entry)
First off, the previous Apache document root for domain1 was at /www/domain1/docs, so the WordPress files wound up like this:
But I also have a lot of dirs that sit alongside of wordpress like this:
We're going to wind up changing that.
Create an subversion external property in /www/domain1/docs for WordPress
$$ cd /www/domain1/docs
$$ export SVN_EDITOR=vi ( or your editor of choice )
$$ svn propedit svn:externals .
vi starts up and you can add the following line:
Save and exit
[code]$$ svn commit
$$ svn update [/code]
( This downloads the WordPress code from the above address into your wp/ directory. Now we are cooking. )
- Now, /www/domain1/docs/wp is where all your WordPress code lives.
- Copy wp-config.php to /www/domain1/docs/wp/
- Copy .htaccess to /www/domain1/docs/wp/
- Create a link from the stock wp-content/ dir to your personal wp-content dir like this
[code]$$ cd /www/domain1/docs/wp
$$ rm -Rf wp-content/ ( use -Rf with care please! )
$$ ln -s /www/domain1/docs/wp-content wp-content
- In Apache config, set document root for this domain to /www/domain1/docs/wp
- Put wp-content/ dir and any other non-WordPress dirs/files into /www/domain1/docs
- Create an alias for wp-content/ and any other non-WordPress dirs/files in Apache config
Alias /wp-content /www/domain1/docs/wp-content
Alias /dir1 /www/domain1/docs/dir1
Alias /dir2 /www/domain1/docs/dir2
This looks like a lot of work, but it's really only a lot the first time around. Next time WordPress has an upgrade:
$$ cd /www/domain1/docs
$$ svn propedit svn:externals . ( change the tag to new version of WordPress )
$$ cd wp/
$$ rm wp-content (to remove the link)
$$ svn update (to update to new version of WordPress)
$$ rm -Rf wp-content/
$$ ln -s /www/domain1/docs/wp-content wp-content
Everything after the propedit in this group can and should be scripted which will basically give you a 2 step process for upgrading WordPress, while keeping you wp-content/ dir under local source code control, as well as leaving room for any other directories or files your site might require.
This technique will probably also work with Symfony although I haven’t tried it yet.
Long term though, this is a problem with my (and possibly your) backup solution. Look here for more info. Specifically this bit:
The certificate chain supporting Amazon S3 SSL is an implementation detail of the system that may change from time to time. A robust application should not depend on the Amazon S3 SSL certificate being signed by a particular certification authority. However, you can depend on the fact that we will only use reputable CAs that are widely supported by existing user-agents. The easiest way to select root CAs to bundle with your application is to simply import the set from a modern web browser with a large market share.