Creole on PHP 5.2.4 with MSSQL

Working in a University or any organization generally leaves you supporting a lot of old code, on even older systems. As part of our web services infrastructure virtualisation project I’ve been bringing a lot of old code on to new systems. As can be seen when I got PHP 5.2.4 running on Ubuntu 12.04 LTS Precise.

As it turns out, our code wasn’t running so smoothly on PHP 5.2.4 as Arthur Koziel points out there is a problem with Creole and PHP 5.2.4. While we didn’t have the exact same problem, his post highlighted where it could be fixed.

Our problem was that we have a third party product using a Microsoft SQL database. It either always had, or just started to spit out date time values in the following format: Nov 14 2012 07:30:00:000PM note the non standard specification of microseconds. (Not sure which way around the problem occurred because we upgraded a month or so ago, but only just got a report of the problem.)

PHP’s strtotime really doesn’t like that format, and I can see why, horrible MSSQL!

I saw that I could fix this in creole/drivers/mssql/MSSQLResultSet.php by providing an alternate getTimestamp function and using a bit of strptime and sprintf magic. Below is the code I used to fix it. This might not be the most elegant solution, but it works!


/**
* @see ResultSet::getTimestamp()
*/
public function getTimestamp($column, $format = 'Y-m-d H:i:s')
{
$idx = (is_int($column) ? $column - 1 : $column);
if (!array_key_exists($idx, $this->fields)) { throw new SQLException("Invalid resultset column: " . $column); }
if ($this->fields[$idx] === null) { return null; }

$ts = strtotime($this->fields[$idx]);
if ($ts === -1 || $ts === false) { // in PHP 5.1 return value changes to FALSE
// in PHP 5.2.4 this no longer works, MSSQL provides date time as
// Nov 14 2012 07:30:00:000PM
$tsUgly = strptime($this->fields[$idx], '%b %d %Y %I:%M:%S:000%p');
$tsNicer = sprintf('%04d-%02d-%02d %02d:%02d:%02d',
$tsUgly['tm_year'] + 1900, // This will be years since 1900, so we need to add 1900.
$tsUgly['tm_mon'] + 1, // This will be the month 0-11, so we add one.
$tsUgly['tm_mday'],
$tsUgly['tm_hour'],
$tsUgly['tm_min'],
$tsUgly['tm_sec']);
$ts = strtotime($tsNicer);
if ($ts === -1 || $ts === false) { // in PHP 5.1 return value changes to FALSE
throw new SQLException("Unable to convert value at column " . $column . " to timestamp: " . $this->fields[$idx]);
}
}
if ($format === null) {
return $ts;
}
if (strpos($format, '%') !== false) {
return strftime($format, $ts);
} else {
return date($format, $ts);
}
}

I hope that helps anyone else out there maintaining old code on newer systems. If anything it’ll help me if I come across a similar problem again!

Steve Daniels

Installing OCI8 on Ubuntu 12.04 LTS Precise (with PHP 5.2.x)

Recently I wrote about installing PHP 5.2 on Ubuntu 10.04 LTS, then again about how I also got PHP 5.2 working on Ubuntu 12.04 LTS Precise in the same way. Well now I’m trying to get OCI8 working in this environment and there were a couple of obstacles.

Firstly, an updated package somewhere along the line broke my fix to libtool.m4. So that needed fixing with:


sudo mv /usr/share/aclocal/libtool.m4 /usr/share/aclocal/libtool.m4.bak
cat /usr/share/aclocal/lt~obsolete.m4 /usr/share/aclocal/ltoptions.m4 /usr/share/aclocal/ltsugar.m4 /usr/share/aclocal/ltversion.m4 /usr/share/aclocal/libtool.m4.bak | sudo tee -a /usr/share/aclocal/libtool.m4

Then you need to follow the manual OCI8 installation method the bit your after starts: “For a manual install, download the PECL OCI8 package”.

Once you get to the following command:


phpize

It’s time to fix things up. The libtool in the package just won’t work when you go to run make. So remove it and link to your local version.


sudo rm ./libtool
sudo ln -s `which libtool` ./libtool

Now you can carry on with the instructions. You may get a few warning messages, but everything should be fine.

Hope that helps,

Steve

PHP 5.2.x with APC on Ubuntu 10.04 LTS Lucid (via Hardy)

As part of a move to virtualise our last remaining physical servers and build more scaling capability and resilience into our web serving infrastructure we have a specific need to run a couple of legacy apps on PHP 5.2.x. In the future when we’ve managed to fit in testing a bug fixes we’ll have these apps running on PHP 5.3, or even 5.4, but for now this need exists.

We have templates for Ubuntu 10.04 LTS, and since this is the middle ground between when PHP 5.2 was last shipped with Ubuntu in 8.04 LTS, and the latest and greatest 12.04 LTS I figured it’d be more secure, efficient and easier to implement it there!

Thankfully most of the work has been done for me.. Randy Fay shows how to run Karmic’s PHP 5.2 on Lucid but Sergius14 suggested pinning PHP 5.2 to the Hardy versions and I tend to agree with the reason. I want the latest security patches bundled in!

So taking Randy’s /etc/apt/sources.list.d/karmic.list and /etc/apt/preferences.d/php file and changing karmic to hardy in the file name and file contents gets us up and running. See the attached files to see what we’re running, if you use these files remove the .txt suffix from the file name.

Because we’re now looking at double the amount of packages, apt is going to complain, we need to increase it’s cache. If you don’t you’ll get an error when you run sudo apt-get update so, following the fine instructions on how to increase your apt cache limit we’ll do this as a one liner:

echo 'APT::Cache-Limit "100000000";' | sudo tee -a /etc/apt/apt.conf.d/70debconf

Next let’s update and install libapache2-mod-php5 to pull in PHP5, Apache2 and the basics.

sudo apt-get update
sudo apt-get install libapache2-mod-php5

Check it’s working by visiting the ip of the server in a web browser.

http://youripaddress/

To check PHP’s working we’ll create a new file calling the phpinfo() function.

echo -e '' | sudo tee /var/www/phpinfo.php

Check http://youripaddress/phpinfo.php to ensure PHP is installed and correctly running. If your browser tries to download a file, you might need to restart Apache to enable PHP.

sudo /etc/init.d/apache2 restart

Now we need APC (There’s a good article on Wikipedia about PHP accelerators, of which APC is one). The php-apc package installable from APT has some unmet dependencies because we’re using Hardy’s packages. We can circumvent that by installing it via PECL instead.

This route isn’t without a few complications because we’re using Hardy’s PHP5.2 some files aren’t in the right places, or automatically included with dependencies.

I added to Randy’s /etc/apt/preferences.d/php the following to get the PHP5 dev files for PECL to run.

Package: php5-dev
Pin: release a=hardy
Pin-Priority: 991

Now we’ll get php5-cli, php5-dev and php-pear (which includes PECL), we’ll update the PECL channel whilst we’re there and then try to install APC with PECL.

sudo apt-get install php5-cli php5-dev php-pear
sudo pecl channel-update pecl.php.net
sudo pecl install apc

AGH!

Errors! As I said before, some files like libtool.m4 and ltmain.sh are in the wrong places and need to be symlinked.

sudo ln -s /usr/share/aclocal/libtool.m4 /usr/share/libtool/libtool.m4
sudo ln -s /usr/share/libtool/config/ltmain.sh /usr/share/libtool/ltmain.sh

libtool.m4 is also missing a few bits since libtool.m4 has been broken down into multiple files so we need to concatenate them back together and append to libtool.m4:

cat /usr/share/aclocal/lt~obsolete.m4 /usr/share/aclocal/ltoptions.m4 /usr/share/aclocal/ltsugar.m4 /usr/share/aclocal/ltversion.m4 | sudo tee -a /usr/share/aclocal/libtool.m4

We also have a missing pcre.h header file. To get that we need libpcre3-dev.

sudo apt-get install libpcre3-dev

This should be it, so lets use PECL to download, compile and install APC (accept all the defaults), enable it in the php.ini and restart Apache.

sudo pecl install apc
echo -e '\n\nextension=apc.so' | sudo tee -a /etc/php5/apache2/php.ini
sudo /etc/init.d/apache2 restart

Now it’s time go back and check http://youripaddress/phpinfo.php to see if APC is enabled!

Congratulations if you’ve read this far and if it helped!

Steve Daniels

Infrastructure dreams

16

I’ll admit it, I like to dream too.. though I’m sure it’s not much of a secret, the content of my dreams will surprise some!

I dream about ideal web serving infrastructures.

There, I said it, and this is my dream:

What I’m hoping to get set up in the new year is all sorts of cool stuff that I’ve been trying to find excuses to implement!

I’d first like Internet users to come in and hit our main Zeus Load Balancer this gives us the flexibility of being able to route the web requests anywhere we like instead of moving IP’s and changing DNS.
Then could hit Varnish Cache and have all the static bits and pieces cached in there which should take load off of poor old Apache (maybe we’ll try out nginx..).
From there I’ll be caching frequently built HTML fragments and database calls in memcached.
I’d like to try and factor in MySQL Proxy at some point to take the load off of out MySQL server and get it spread out to some slaves, which should make backing up much quicker too.

What’s your infrastructure like?

Can people see and problems and suggest alternatives?

Ste Daniels

P.S. I totally published this on the 16th at 8am and not on the 17th at 4:30pm! (Maybe the snow delayed the delivery?)

Dashboards

Colorful house number, two
In my previous post I talked about Wallboards. As an aside, I’ve just started experimenting with using some old O2 Joggler’s as miniature information radiators in my new home. But that’s another blog post for another blog! This time round I’m covering dashboards!

Dashboards are beautiful things that seem to be getting slung haphazardly into all sorts of applications these days.. “view this on your dashboard” “view that on your dashboard” “configure it from your dashboard”.. blah blah blah. Dashboards can be a Good Thing™.

Here at Edge Hill University we run the our GO portal, we’ve written about GO many times so I won’t go into detail here, suffice to say it’s a portal that gives you tabs – your Home tab is effectively your “dashboard”. On my GO Home tab I’ve got some useful information shown to me – it stops me hunting around various places, all of it is Edge Hill specific.

GO home tab

On there I’ve got mail, Edge Hill rss feeds, staff directory, GO news, the forum – all bits of information I regularly check. The dashboard pulls it all together so I can access it at a glance. Saving me a lot of time.

I do the same with JIRA and Fisheye as you can see below:

JIRA dashboard

All my development information needs at my fingertips…

Do you pool information you want on dashboards of some sort of other? I know I’m missing the all important personal non worky dashboard for Facebook, Picassa, Flickr and other news feeds.. but I’m at work – I get distracted easily enough as it is without all that!

How many dashboards do you have? What’s on them?

Next up is home tabs..

Ste Daniels

Wallboards, dashboards and home tabs!

Colorful house number, twoWhat do you have on yours?

I’ve been avidly watching The Ultimate Wallboard contest that my favourite productive development software maker Atlassian sponsored.

Taken from the site:

A wallboard is a type of information radiator used for extreme feedback. Wallboards are typically displayed above a development team’s workspace.

Now that’s got you wondering.. what’s an “Information Radiator?”:

An information radiator is a large, highly visible display used by software development teams to show anyone walking by what’s going on.

Wallboards can be beautiful catchy looking things, and they should be!

As you can see, wallboards can be quite complex things, that take large amounts of important information, prioritise it and keep it visible to the people who need it most.

We want one at Edge Hill Web Services, we use JIRA here so can easily pull data out of that and I’m sure we could squeeze some information out of RMS while we were at it too, albeit with a LOT more hacking (software like that SHOULD have an API!). Looking to the future we’ve just got Crucible and Bamboo licences so that information would join out mythical wallboard..

Do you have a wallboard? / Do you want one?
Is it useful? / Can you see it’s potential use?
What’s on it? / What would you have on it?

You’ve got me every Thursday till Christmas so stay tuned for the next in my series.. Dashboards or home tabs!

Ste Daniels

ERROR 2013 (HY000): Lost connection to MySQL server during query

If you’ve just set up a new mysqld service and your getting the above error when trying to connect over TCP (even when tunnelling in over SSH), don’t fret. The answer lies in /etc/hosts.allow. You need to add in a line similar to this:

mysqld: 192.168.0.123

This foxed me a while ago and I didn’t get round to fixing it as it wasn’t yet mission critical, however it also stopped us dead in our tracks yesterday. Thank fully to Neil in Core Services who had a moment of clarity we’re now back in business.

UPDATE:

Now I know the problem, a little more simpler Google querying led me here a 2004 post on MySQL’s access denied troubleshooting page which mentions that the cause of this is mysqld being compiled with tcp-wrapper support.

Winter bugs

Here at Edge Hill we’ve used Trac for quite some time in Web Services. I’ve been here for more than a year and a half now and during this period the team have greatly increased their use of Trac for ticketing problems, monitoring source changes, and more recently tracking Scrum Sprint progress. While Trac is great we need more – we want more – we’re bias.

We use Confluence heavily, the Faculty of Health have it as home to many documents and information for staff, students and external partners, we’re moving our Intranet into it, we also use it as our team wiki for documentation and the like. So as you can see, we have a fair bit invested in it. Confluence is made by Atlassian and these great folks also make JIRA.

We’ve heard many great things about JIRA and seen it in action ourselves when dealing with Atlassian. We think it’s great, has good integration and with the GreenHopper it’ll help us with our Scrum agile project management!

That’s the plan anyway!

Whilst we’re at it we’re taking the opportunity to install FishEye too. This will give us some awesome insight into what we’re actually coding and allow some great collaboration and knowledge sharing without much effort at all. If your a stats junky the numbers and graphs this thing makes is awesome!

Do your use JIRA?

One project to rule them all (ala Second Life’s Linden Labs) or many little ones?

What are your favourite things about FishEye?

Your Wiki!


In an earlier post on our wiki I mentioned that we use Confluence from Atlassian this company has a good and steady release cycle. Currently we’re couple of versions behind the latest, but hopefully that’ll soon be addressed by a move to a new much faster server!

Some of the fixes and new features that we’ll get when we upgrade are:

  • The Widget Connector – a quick an easy way to embed content from Youtube, Flickr, etc etc.
  • Various fixes and upgrades to the Rich Text Editor
  • The Macro Browser – this is an awesome new tool that is part of the RTE that allows you to browse for useful macro’s to add features to your confluence pages in a quick and easy way
  • Various performance issues have been solved
  • The Confluence PDF export has been drastically improved, especially that you can provide a PDF CSS stylesheet
  • Office 2007 files are now fully supported, searchable and embeddable in the latest version of Confluence which is just great!

Is the wiki being used?

We’ve had over 20,000 page views in the past month and we still haven’t fully launched the system to the university at whole! That being said, the Faculty of Health whom I work for have been using it extensively over the past year at least.

A quick breakdown:

graph

Currently we’ve got about four and a half thousand pages in the wiki, with about sixteen thousand versions of these pages. That averages out to about four and a half versions of each page. There are also almost ten thousand documents attached to pages on the wiki.. Good work on editing people!

Wiki Meetings


Early on in the New Year the old Intranet here at Edge Hill will be abolished. All of its content will be available directly through GO. The platform we’re hosting this content under is called Confluence and it’s a wiki.

To the majority of people this won’t mean much right now but we’re hoping in the future it will. You’ll be able to manage your own content on GO for dissemination to colleagues you work with on a daily basis and those you work with across the university.

This new way of managing things will hopefully reduce the amount of out of date information that is out there. You shouldn’t be emailing around Word documents within the university. Because the moment you send that document it’s out of date, your recipient just doesn’t know it yet!

Lets take Meeting Agenda’s as an example. The below video describes why you shouldn’t be emailing meeting agendas and why you should start using a wiki:

The Faculty of Health is already doing stuff like this, posting meeting dates in their calendar, having agendas online, with misc documents for the meeting available on each meetings dedicated pages. You department can too!

Another brilliant use of the wiki is to keep track of action points:

These excellent videos are courtesy of Stewart Mader’s 21days you can find a lot more videos there and other extremely useful wiki related resources.