Arrays in dBs and YAML Config

I’m working on some new mega-menus for GO. I’ll post about that when we go live.

Following a pre-Christmas code review of the user-defined munu choices, we identified that we needed a way to provide a default set of menu items for everybody prior to them making and saving their personal choices. Mike remembered that we had an Option table capable of storing an array of values per person.

As this table existed in the GO symfony framework it already had methods for getting and setting values. Reading those methods introduced me to the php serialize function. This function takes an array and generates a storable representation of its value. For example this
   [0] => 1
   [1] => 2
   [2] => 3
   [3] => 4
   [4] => 9

is turned into this:a:5:{i:0;i:1;i:1;i:2;i:2;i:3;i:3;i:4;i:4;i:9;}

When the value is queried we can simply unserialize it to return it to its php readable form.

Default Values in YAML Config Files

In the Option table, if a record isn’t returned for a user (no option has been set) a default value (or array) can be returned when passed into the querying method. As I needed to query the data in more than one place in the application, I needed to set the default value in one place. Symfony uses YAML files for configuration so I set it in the application config file app.yml.
    default_dashbar:      [1, 2, 3, 4, 9]

Spaces are all important in YAML. Ensure you don’t indent each line and include spaces between array elements.

To pull YAML config variables into your code enter:sfConfig::get('param_name', $default_value);. The ‘param_name’ takes the file name and the parent child structure of the YAML file, separated by underscores and can be found in the Configuring Symfony chapter of the manual.

Heavy duty stuff for the first post of 2011.

More calendars to watch

We’re not the only kid on the block this December so be sure to check out some of these other advent calendars:

24 Ways

If web design is more your thing then you could do worse than following 24 ways for the next few weeks. Variety of well known web folks share their ideas as blog posts… Kind of like the Web Services blog but without the well known bit!

Equally useful is the archive right back to 2005 so you’ll have stuff to read into the new year.

University of Warwick

This will be particularly interesting to watch because I thought about doing an advent calendar for main Edge Hill website a few months ago but it didn’t get off the ground in time.  Ideas for next year though!

Symfony Advent Calendar

The original symfony advent calendar, published in December 2005 was one of the key things in persuading me to adopt symfony for the Education Partnership website.  Symfony now forms the core of the corporate website, GO and Hi – it’s adoption at Edge Hill has radically changed the way we build websites.  This year the symfony project have created a brand new advent calendar and I look forward to following it.

Newby’s Advert Calendar

Binary Calculator from Newby's Advert CalendarIf you like surreal comedy advert calendars satiring local newspapers then there is literally only one advent calendar for you this year! The Framley Examiner created this quite a few years ago but I still love it.

There’s a few I’ve come across – post a comment to tell us about any other cool online advent calendars, or even just to tell us what the one on your wall is like.

Building an Anti-CMS

At last weekend’s PHP North West Conference I delivered a talk titled Building an Anti-CMS (and how it’s changed our web team).  Feedback has generally been pretty positive so I thought I’d open it up to a bit of constructive criticism from inside the sector (because every web team reads our blog, right?!).

Video from the talk itself is due out within the next month but I re-recorded some audio to turn it into a slidecast to make it a bit more useful:

I’ve given a number of talks before at Edge Hill, at BarCamps and at IWMW but for PHPNW I’ve tried to further develop my style of presentation. Over the last 6 weeks I’ve watched quite a few “Lessig style” talks – making use of lots of short sentences and pictures and not being afraid to have nothing on the screen.

It leads to a massive slidedeck – 86 slides for 13 minutes – and there’s far less room to ad lib but it gets away from some of the things that annoy me about regular death by powerpoint. I’ll let you make up your own mind whether it’s worked!

Faded tag cloud

It’s been a little while since I posted anything techie, and a while since I posted anything that might be of use to the symfony community so I’m going to post about the exciting topic of tag clouds!

Andy has posted about tags before so read that for more information about the concepts. We’ve had tag clouds on the site since March but with the latest designs Sam wanted something a bit more stylish. Someone came across a site with a faded tag cloud where colour rather than size determined the weight of the tag so Sam built that into the new designs.

We use a symfony plugin catchily called sfPropelActAsTaggableBehaviorPlugin which allows us to add tags to any object taken from the database (e.g. courses, events, news articles). The plugin also has some functions to output tag clouds for different types of objects and it’s this we used on the old site.

For the faded tag cloud I was planning to implement it from scratch but then wondered if it was possible to do something with the existing taggable behaviour plugin. It was, and here’s how!

Firstly we modify the output of the getPopulars() function to sort by weight:

$c = new Criteria();
$c->add(TagPeer::IS_TRIPLE, false);
$c->add(TaggingPeer::TAGGABLE_MODEL, 'newsArticle');
$c->setLimit(isset($this->limit) ? $this->limit : 30);
$this->tags = TagPeer::getPopulars($c);

Then in the template we simply output the tag cloud as normal:

<?php use_helper('Tags'); ?>
<?php echo tag_cloud($tags, '@news_tag?tags=%s'); ?>

This will produce a cloud where the size determines weight of tag with the largest at the top, which isn’t what we want, but the clever stuff can be done with CSS. The plugin produces an unordered list with <big> and <small> tags used to change the size of the tags. We use the following CSS to remove the font size styling and apply color (sic) to the text:

ul.tag-cloud li big big,
ul.tag-cloud li big,
ul.tag-cloud li,
ul.tag-cloud li small,
ul.tag-cloud li small small{font-size: 103%}
ul.tag-cloud li big big a {color: #EFEFEF}
ul.tag-cloud li big a {color: #BFBFBF}
ul.tag-cloud li a {color: #8F8F8F}
ul.tag-cloud li small a {color: #5F5F5F}
ul.tag-cloud li small small a {color: #3F3F3F}
ul.tag-cloud li a:hover{ color:#fa0}

That’s about it – a nice easy way to produce good looking tag clouds thanks to the wonders of symfony and CSS.

Choice Part 6: Lucene in the sky with diamonds

Search is one of the key ways that visitors find what they’re looking for on our websites. A good search engine can quickly and acurately direct the user to the right place and make for a more efficient and productive experience.

In the past we’ve used Novell’s QuickFinder search service to spider the site, supplemented by a couple of custom search systems for things like courses. I’ve never been entirely happy with the results that QuickFinder provides.

Recently in Higher Education and beyond, there has been a trend towards Google’s search appliance and their hosted solutions. Both of these are excellent in terms of raw power – they will happily index every page on a site and searches are quick and mostly relevant. But there’s more to a good search engine than the size of the index – they must provide the results you’re looking for and present them in an easy to understand way. Here’s a fairly typical example of the top search result for a search for “Computing” (I’ve removed identifying names!):

The University of Somewhere

For Edge Hill it’s important that prospective students are able to find what they’re looking for. So in the above example it’s good that it has picked a page about the academic department rather than what at Edge Hill would be IT Services, but it’s actually the Faculty page giving the briefest of details. The summary doesn’t help at all – the spider has picked up details from the page header including the alternative text from the logo and the breadcrumb trail.

What we want are relevant results which allow the visitor to quickly identify what pages have been found with information that’s relevant to the results, not just scraped text. Some search engines are starting to do this – when Google finds videos it will show a thumbnail and allow you to play the video inline – so we can use some of these ideas when creating our own search system. Now let’s get a bit more technical!

Our website can be split into two types of information – structured and unstructured. When I say unstructured, I don’t mean that it’s hundreds of pages put online without any consideration – I’m talking about web pages of content that aren’t stored in a database. Structured information is pulled out from one of our databases – things like news, events or courses. Structured content is what most search engines find difficult because they don’t “know” what a page is all about, but we do, so we can tell our search engine what information is important and how we should represent it.

For our new website, we’ve introduced a new search system based on Zend Lucene. Lucene isn’t a full blown search engine, but it’s a library you can build on to provide full text indexing of almost anything you want. We’re using a symfony plugin which packages a lot of search functionality to allow us to index news, events, courses and other information directly from the database. We have control over what information is indexed for each type and the weightings applied to them. For example we give courses a slightly higher weighting than news.

For static content we have a custom spider which trawls all the other pages on the site and adds them to the index. This work like any other search engine, following links and determining which text is relevant. We try to exclude the header, footer and navigation from the index as this contains text which is common to many pages and adds little to the value of the page.

Edge Hill’s computing search resultWe can also do a lot with the search interface itself. Firstly, different types of result show different information. For example a course result shows the UCAS code, qualification, which campuses it runs at and allows the course to be added to the My Courses basket for comparison. News and events shows similar custom results while static pages show the usual snippet of text from the page, but without irrelevant text from outside the content area creeping in.

Overall the new search seems to be working quite well – we’re able to embed it into the rest of the site more than we’ve done in the past and provide custom search boxes for courses and news. There’s still work to do on it though to improve the accuracy of results, so if you’ve tried the search and not found what you were looking for easily, please let us know.

Choice Part 2: A new platform

For years, most of the corporate website has been produced as static web pages using Dreamweaver. This has worked well – we’re able to ensure the design is tight, content correct and the site doesn’t grow to an unmanageable size.

To help manage content on the site, Web Services have produced a few key applications – news, eProspectus, job vacancies for example – and while they’ve worked great for each area, integration with the corporate site hasn’t gone much further than matching the template and manually linking between the static content and the dynamic applications.

With the Big Brief we’ve had the chance to build dynamic web applications into the core of the website. Instead of being an add on, our main website is dynamic and existing content is linked in. This allows pages on the site to fully embed content – for example we can have news and events on the homepage, or Faculty sites can list the courses they offer.

Symfony news roundupFor the corporate website we’ve extended our use of the symfony web framework. We’ve been using symfony for about 18 months, first for Education Partnership, then the Hi applicant website and the GO portal. I’ve posted before about some of the advantages it gives us, but it’s developed significantly over the last year and we’re starting to really see some of the benefits in terms of consistent coding standards, making use of plugins so that we’re not reinventing the wheel and allowing us to rapidly build new systems that integrate with the rest of the website.

Introducing symfony to the core of the corporate website is just the first step in making it more dynamic. We’re working on allowing visitors to the site to login to gain access to more personalised information, not just for staff and students, but for applicants, partners and other users of the site. The applications we’ve developed will allow dynamic content to be spread around the site – for example courses for a department, news feeds or relevant events.

In the next couple of posts we’ll be talking about some of the applications you’ll see on the site and then maybe, just maybe, I’ll talk some more about future plans!

PHP London 2008

A belated writeup on last week’s PHP London Conference. Andy’s already written a post so I don’t feel too bad!

As it turned out we split the sessions so I’ll just cover those Andy’s not mentioned. First up was Stefan Esser‘s PHP Binary Analysis. It was looking at using complied PHP bytecode to debug and audit your code. Probably of more use to people doing detailed security audits but some interesting ideas that I’d like to look into when I get a bit more time.

After lunch Marcus Bointon presented Mail(); & Life after Mail(). He started early on by quoting a blog post from Hacked:

I Knew How To Validate An Email Address Until I Read The RFC

Anyone’s who’s ever tried to send email using PHP’s mail() function will know the lengths you go to to get things working. Even then you’re probably doing it wrong. The solution is to use a library to handle all the standards compliance for you, something that symfony provides through the PHPMailer library.

Marcus went through a bunch more libraries and compared some of the features they provide so it will be interesting to look into what’s best for our needs.

More interesting for me was finding out about return paths. This is what happens when an email bounces and with a bit of server side magic it is possible to handle errors better. It’s quite a complex task to do properly so I’m interested in a good hosted service which can be used for both one shot emails like user registrations for batch mailshots. Apparently there’s a few services out there but I’ve not seen any with a really good API.

Final session I went to alone was My Framework Is Better Than Yours? presented by Rob Allen, Toby Beresford and Ian P. Christian. Each gave a short presentation on their framework of choice – Zend, Code Igniter and symfony – followed by a panel discussion. It was clear that each has its advantages and disadvantages:

  • Zend is good for components to pick and choose which aspects of a framework you need. It can often be used with other frameworks too. This can also be a downside is they’re maybe not quite as integrated as other systems.
  • Code Igniter is lightweight and some might like that it runs under PHP4. Personally I think this is a disadvantage. Someone in the audience suggested there was a way of turning on HP5 mode but I can’t believe this does more than activates a few extra features. Coding for PHP5 is an attitude shift and I don’t see how they’ve done this while retaining compatibility.
  • symfony, well I knew a bit about that already 😉 Pookey did a pretty good job of presenting it.

During the panel discussion there was a comment about the criminal use of the term MVC to describe the frameworks. It got the attention of the room and there’s quite a lot of talk about this on the interweb. My view is that it doesn’t really matter whether a framework sticks rigidly to some design pattern if it provides the features that you need. I’m interested in getting things done, not in the theory of system design.

That’s all from me – check out Andy’s summary of the other sessions.

One Big Summer (of Code)

Google have announced another Summer of Code. This is a program [sic] to let students work on open source projects over their summer holiday (vacation) while getting paid for the work they do.

Over the last three years, 1500 students have worked for scores of mentoring organisations to add features, fix bugs and improve products for the benefit of the community. Looking the 2007 website you can the sorts of work being done. Everything from Haskell to PHP, applications like Gallery, Drupal and WordPress.

This year symfony – the web framework we use extensively within Web Services – has applied to join Summer of Code and it will be great to see what comes out of it.

Applications for students open later this month and it looks like UK university students would qualify so if there’s any one out there in Edge Hill reading then keep an eye on the website and see what projects turn up. It’s not restricted to Computing students although you will need some coding skills (obviously!).

Web Services Project Officer

Web Services are recruiting a new member of staff. We’re looking for someone to join our team to work with the Faculty of Health to create a system to link staff, students and external partners to aid communication and better manage information.

The first half of this twelve month contract will focus on planning and implementing an extranet so we’re particularly interested in people with experience of wikis, content or document management systems. Later in the year the successful applicant will be working on other projects for the Faculty to help improve communications.

You can find the full job description and person specification on the website but if you have any questions, please do not hesitate to contact me on or phone 01695 584195.

Bad URLs Part 4: Choose your future

So we’ve looked at examples of bad and good URLs, what the current situation is at Edge Hill, but what are we doing about it?

Lots, you’ll be pleased to know! As part of the development work we’ve been doing for the new website design, I’ve been taking a long hard look at how our website is structured and plan to make some changes. There are two areas to the changes – ensuring our new developments are done properly and changing existing areas of the site to fit in with the new structure.

Firstly the new developments. We’re currently working on three systems – news, events and prospectus. News was one of the examples I gave last time where we could make some improvements so let’s look at how things might change.

Firstly, all our new developments are being brought onto the main Edge Hill domain – – and each “site” placed in a top level address:

News articles will drop references to the technology used and the database IDs:

In this example the new URL is actually longer than the old one, but I can live with that because it’s more search engine friendly and the structure is human-readable. For example we can guess that the monthly archive page will be:

This idea is nothing new – for the first few years of the web most sites had a pretty logical structure – but it’s something that has been lost when moving to Content Management Systems.

The online prospectus is getting similar treatment where courses are currently referenced by an ID number the URL will contain the course title:

As part of our JISC funded mini-project, we’ll be outputting XCRI feeds from the online prospectus. The URLs for these will be really simple – just add /xcri to the end of the address:

In the news site, feeds of articles, tags, comments and much more will be available simply by adding /feed to the URL. Same will apply to search results.

All this is great for the new developments, but we do have a lot of static pages that won’t be replaced. Instead, pages will move to a flatter URL structure. For example, the Faculty of Education site will be available directly through what is currently the vanity URL meaning that most subpages also have a nice URL:

Areas of the site which were previously hidden away three or four levels deep will be made more accessible through shorter URLs.

How are we doing this? The core of the new site is a brand new symfony based application. This allows us to embed our dynamic applications – news, events and prospectus – more closely into the site than has previously been possible. symfony allows you to define routing rules which while look complex on the backend because of the way they mix up pages, produce a uniform look to the structure of the site.

For our existing content we’re using a combination of some lookup tables in our symfony application and some Apache mod_rewrite rules to detect requests for existing content. All the existing pages will be redirected to their new locations so any bookmarks will continue to work and search engines will quickly find the new versions of pages.

That’s all for this little series of posts about URLs. Hopefully it has helped explain some of my thinking behind the changes. If you’ve got any questions then drop me an email or post a comment.