Site Confidence

55% of sites don’t use Cache-Control: max-age

noreply@blogger.com (Site Confidence) — Mon, 30 Apr 2012 08:43:00 +0000

Whilst referencing the excellent HTTP archive recently I came across this rather interesting little stat - http://httparchive.org/interesting.php#max-age.

Over 50% of the assets served by the sites that are crawled by the archive, do NOT set a Cache-Control: max-age value.

My initial thought was that this is pretty shocking really.

When I actually thought about it further however I realised that in all of the Health Checksthat we run for customers, this is something that comes up time and time again.

One of the most common recommendations that we make to site owners / developers is to set a ‘far future’ expiration date for static content, or content that is unlikely to change for some time.

The benefits are twofold (and well documented) – repeat visitors get a much faster experience and the sites saves on bandwidth costs because it’s serving less content.

So why is this value so high? Should we be surprised that it IS so high?

It possibly serves to highlight a lack of knowledge or understanding around cache control in general. We often see it (it’s fair to say that over 50% of the health checks that we have completed have made this recommendation in some form or another), so perhaps further education is the solution?

Given all of the ‘noise’ generated by WPO in the recent months I was sure that this number would have considerably reduced since Steve first posted this.

It’s reduced by 1%.

Why Histograms are cool!

noreply@blogger.com (Site Confidence) — Mon, 23 Apr 2012 10:08:00 +0000

Quite a while ago we added Histograms into our monitoring portal, and ever since then I’ve always found them to be incredibly useful!

I was recently talking to a prospect (who didn’t have as much of a passion for stats as we do at SC Towers) about the value of Histograms and they were questioning in what situation they should be using them. We had been talking about longer term analysis of performance data, specifically identifying when and to what degree their page performance varied from the average.

Quite naturally, they had been using the Daily Average report, showing mean, min and max values. This report is useful but the one thing it doesn’t allow you to investigate is the spread of the test results which had strayed beyond the mean.

Let me give you an example.

Using the Daily Average Speed report to look at performance data over a 7 day period gives you the following:

This gives a good indication of the mean variation throughout the week, and shows us just how bad (by bad I mean how far from the average) some of the results have been, but it doesn’t tell us how many of the results were on or near to the max.

By using a Histogram we can determine exactly what the spread of results is. The histogram below was generated for the same page monitor over the same time period.

Here we can see that although the majority of test results have been placed in the 8-9 seconds ‘silo’, which is in line with the average reported above, almost 40% of the total number of results returned are outside of this average (either because the page returned quicker than expected or because it was much slower to respond with a full page load).

The gamma distribution on the histogram below is much more severe, but could quite easily be produced using the same or similar data from the Average Speed report at the top of this blog.

This shows us that averages can hide a lot of sins!

So, why are histograms cool? Well, in my mind anything that gives me some extra information is cool. So the fact that these histograms have shown me the variance in these test results is a good thing.

They’re also cool because they can show things like this. The Histogram below is definitely what you don’t want to see being produced by your platform, as it is highlighting massive performance variations, in this case at different times of the day. During the quiet periods the site is relatively stable, but then as things get busier the site starts to slow down resulting in the ‘rock horns’ effect.

Some Rock Horns!

A Histogram!

And anything that makes the rock horns is cool by me!

80/20 Follow up observations

noreply@blogger.com (Site Confidence) — Mon, 02 Apr 2012 10:35:00 +0000

A few weeks ago we looked at the 80/20 argument as presented by Steve Souders, and reasoned that the backend time might actually be greater if you take the response times into account for every object on the page, rather than just the initial request.

Of course, as Pat Meenan rightly pointed out, it should be about understanding the full picture (front and backend) of your own performance. To my mind that full picture extends to identifying performance bottlenecks across the entire page download.

In last week’s blog post we showed how the backend response times can amount to a significant number, skewing the 80/20 assumption. This quick calculation was based on those backend times being consistent.

We have observed that those requests are not always consistent, and that variance at the object level will have a knock-on effect on total page downloads and therefore user experience.

In the example below, the data start time for a single static asset served on a simple page is varying between 0.20 and 0.40 seconds. This may not look like much but given that (for example) the average number of images served per page is currently 54 (http://httparchive.org/trends.php#bytesImg&reqImg) this variance can start to mount up.

In this instance we can see that the data start time (and to a lesser extent SSL connection time, although at least this measurement is consistent) is causing the slow object download time.

Sometimes the cause of the variation isn’t quite as clear however.

In the 2 waterfall graphs below, some of the components on the page seem to take a varying amount of time to download for each test. For example, at times bannerInfo-bg.png takes 1.2 seconds, and yet earlier in the day it only took 0.102 seconds.

Sample Waterfall 1

Sample Waterfall 2

We can also recreate this in the real world using HTTP Watch.

First Test

Second Test

By far the largest variation is for the Receive time. Over 0.647 difference to download an object that is under 3Kb. What could be causing this variation?

So this has now become a great example of when it would be useful to start cross-referencing internal measurements (e.g. from APM solutions to determine how long it takes the application to start sending the data or from network tools to try and understand why some objects take longer than expected to be transmitted) against external performance monitoring data.

Armed with this end-to-end view we can build up a picture and some understanding of the performance of systems serving nominally static content and whether it is truly variable in and of itself or whether it appears to be affcted by unfavourable external conditions such as the poor performance of intervening networks.

Although the real world test was conducted using a local office network where there could be local contention, it has been repeated countless times in the ‘clean room’ environment of the monitoring service as well, so we should be able to discount local causes as the reason for the varations.

The important take away from all of the examples above is that you need to have full visibility of the entire application delivery process, and that without accurate measurements you have no idea of a) the current situation and b) if optimisation work / bug fixing /general maintenance is having a positive or negative impact.

Whilst there is a danger of getting carried away with just the numbers (one of Deming’s “7 Deadly Diseases of Management” was to run a company on visible figures alone- http://en.wikipedia.org/wiki/W._Edwards_Deming) measurement has to be one of the cornerstones of good web performance management.

Querying the 80 / 20 rule

noreply@blogger.com (Site Confidence) — Wed, 14 Mar 2012 10:39:00 +0000

First of all, some required reading. Go here and read this, then come back.

Essentially my pitch is this. It is not 20% backend, it is 20% response to initial request time. That’s not quite as catchy granted but it depends on how you look at it.

Whilst I accept that the definition of ‘backend’ time is generally the “time it takes for the server to get the first byte back to the client”, I don’t understand why this terminology should only be applied to the initial request. In every instance when a page is required to download additional content, the application has to carry out some work, and deliver this content to the client. This is additional backend time, it’s just not the initial backend time.

Of course, for sites that are dynamically generating content on the fly there will be a more significant overhead – but even sites that serve static content (mostly images, but JS and CSS files aren’t usually dynamically generated per user either) can present an element of back end processing time, and this time should be taken into account when discussing the 80 / 20 rule. I still agree with Steve’s quote that “… the longest pole in the tent was frontend performance…” but there also needs to be some perspective around this. If the backend cannot serve the content quickly enough then any optimisation work that has been conducted will be, if not in vain, then at least not as effective as it perhaps could have been.

Furthermore, plenty of systems serve “static” objects (JS, CSS , images etc.) out of a CMS, and it may well take time to either locate the contents or retrieve them from database / remote file systems etc. All of this will look like backend time to the client, and should be considered so given that it lies within the control of the host.

How best to demonstrate this? With a picture!

This is an excerpt from a waterfall graph for www.amazon.com. The initial request time includes 0.21 seconds of DNS lookup time, 0.07 seconds of connection time and 0.13 seconds of data start time. That’s 0.41 seconds of ‘backend’ time before the content starts to load. Given that the total load time for this page was 4.7 seconds that’s apparently 90/10 in favour of frontend time.

However, almost without exception every other object beneath this initial request also carries elements of connect time and data start time.

Whilst it would be incorrect to stitch all of these separate backend times together (given the parallel download of many of these objects) I can have a go at roughly calculating the processing time by selecting the first individual object from each ‘wave’ of parallel object downloads. The total recorded backend time is now approximately 3.25 seconds.

90/10 has now become more like 70/30 in favour of backend – we now have a shorter pole!

Here at Site Confidence we often debate the true meaning of the 80/20 rule. If all ‘static’ content served by your website is served how static content was born to be served then brilliant – go sort out your 80!

However, if you’re stuck with a legacy CMS, an unusual CDN implementation or simply have never paid that much attention to your strategy for serving static content you’ve still got 2 poles to shorten.

The 80 is very important, and we’d never argue with Steve about this but that ‘20’ might be bigger than you think.

We're hiring!

noreply@blogger.com (Site Confidence) — Mon, 12 Mar 2012 16:36:00 +0000

Site Confidence are always on the lookout for talented people and right now we've got quite a few positions to fill.

If you are interested in any of these then please get in touch with Steve using the contact details at the bottom.

If you think you know of someone then please pass these details on.

Professional Services:
1 x Senior Technical Consultant: http://www.blueglue.eu/job_detail.php?jid=1557

Join our passionate (and growing) performance team!

Development:

1 x Senior PHP Developer / Architect: http://www.blueglue.eu/job_detail.php?jid=1427

2 x PHP Developer: http://www.blueglue.eu/job_detail.php?jid=1511

Web Operations:

2 x Linux Systems Administrator: http://www.blueglue.eu/job_detail.php?jid=1535

Sales:

1 x Business Development Executive: http://www.blueglue.eu/job_detail.php?jid=1417

We're looking for focussed New Business Development Managers to help us spread the word. Ideally you'll have sold SAAS with a strong background in solution sales.

The man to contact is Steve James - drop him an email or call him on 01183 094 206.

So what does happen to the Apple store on big announcement days?

noreply@blogger.com (Site Confidence) — Fri, 09 Mar 2012 14:45:00 +0000

Following on from Josh Bixby’s recent observation of the apple.com home page through Webpagetest (http://bit.ly/wLhWFY), I thought it might be handy to take a slightly more detailed look into exactly what happens to the Apple UK Store when they make their big announcements

The following performance report shows the time taken for a full page download on http://store.apple.com/uk for the period 7th – 8^th March.

Given that the announcement of the “new iPad” was made at approximately 6 p.m. (UK time), this report shows that something significant was happening to the site as early as 15:00. The fact that it happened pretty much bang on 15:00 could suggest that this was intentional behaviour – however the actual error displayed would suggest otherwise. From 15:00 to approximately 19:45 the site was consistently returning an HTTP 503 (Service Unavailable).

Request 

GET /uk HTTP/1.1

Host: store.apple.com

Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*

Accept-Encoding: gzip

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-gb; SiteCon/8.8.14)

Header 

HTTP/1.1 503 Not Found

Date: Wed, 07 Mar 2012 19:41:56 GMT

Server: Apache/1.3.41-ps_webdav_01 (Darwin)

location: http://store.apple.com

Last-Modified: Wed, 07 Mar 2012 11:08:01 GMT

ETag: "4d89a-e080-4f574191"

Accept-Ranges: bytes

Content-Length: 57472

Content-Type: text/html

x-frame-options: sameorigin

Pragma: no-cache

That’s it, no helpful error messages or fluffy fail whales here.

In some of these instances the 503 wasn’t even returned immediately, users were expected to wait for a period of time for the back end to ‘catch up’ before the error was presented to frustrated fanbois.

In the graph above, the light blue element represents Data Start time, the time taken for the first byte of data to be returned to the client.

This wasn’t an isolated spike either. Here is a report showing an accumulated view of the page download presenting DNS, Connect, SSL Connect, Request Sent, Data Start and Content Download as individual metrics.

The store page came up very briefly between 20:00 and midnight, but more often than not users were instead presented with the familiar “we’re busy updating the store” page.

This wasn’t consistent however. Sometimes the store page was displayed giving users a tease of the new fondle slab but again the back end times may have put off all but the most determined users.

So why is this important? Well as far as I’m concerned, given the almost limitless resources available to Apple I’d have liked to have seen them handle this promotion a little more gracefully.

Their goal should at the very least have been to ensure that the performance graph stayed green, which would have meant not returning HTTP 503 error codes and not allowing poor back end response times to affect the end users experience. After all, Apple have been rather busy of late promoting their cloud services to their customers so perhaps they should have thought about temporarily pointing their users at some of this elastic capacity in order to retain a seamless experience?

In Apple’s case I’m sure that their brand won’t have been damaged by this. Their users tend to be rather forgiving.

Other sites may not be as lucky.

IE9 improves caching of re-directs but doesn’t (currently) play nicely with HTTP Watch

noreply@blogger.com (Site Confidence) — Thu, 08 Mar 2012 14:03:00 +0000

HTTPWatch is very much a part of our toolkit, and we regularly use it to help diagnose and troubleshoot our customers' performance problems. It’s widely regarded as one of the best browser plugins around for client performance analysis.

During some initial analysis prior to conducting a health check we noticed certain unexplained caching behaviour through HTTP Watch in IE9.

It looked like initial page requests were always being cached, even though we had opted to clear cache and cookies through HTTP Watch.

A little bit of digging around led us to http://blogs.msdn.com/b/ie/archive/2010/07/14/caching-improvements-in-internet-explorer-9.aspxwhich describes IE9’s caching improvements. The specifically interesting bit here is:

Redirect Caching

Internet Explorer 9 now supports caching of HTTP redirect responses, as described by RFC 2616. Responses with Permanent Redirect status (301) are cacheable unless there are headers which forbid it (e.g. Cache-Control: no-cache) and those with Temporary Redirect status (302 or 307) are cacheable if there are headers which permit it (e.g. Cache-Control: max-age=120).

To validate this finding we took a look at a trace from an IE9 browser that we knew hasn’t been to the site. Here, we can see this holds true with what is written above, in terms of IE9 caching behaviour:

We can see with both the 301 redirects, there are no cache headers preventing the redirects from being cached.

We then cleared the cache and cookies via HTTPWatch and went to the page again, and got the cached redirects, as we had seen previously:

Our next step was to try clearing the cache directly from IE9, which worked like a charm and cleared the redirects.

So, it does look like the HTTPWatch call to clear cache/cookies does not include clearing cached redirects.

We’ve raised this with Simtec who are going to look into the issue.

3 new reports available in the Management Reporting portal

noreply@blogger.com (Site Confidence) — Thu, 09 Feb 2012 16:17:00 +0000

Once again, you asked and we listened.

We’ve had several requests over the last few months to add a few extra reports into the Management Reporting portal and last week they cleared testing and were pushed live! They are as follows:

Uptime by URL (3 Monthly)
For those of you who like to produce your URL availability reports quarterly!

Uptime by Account (3 Monthly)
Same idea as above but with account wide availability.

Availability and Performance Summary
Finally a rather clever summary report showing availability over time (with a corresponding trend) and some average performance figures.

As with all these things, if you want a more detailed demo then feel free to get in touch.

New GA Stats – Excluding Site Confidence monitoring from your user analytics

noreply@blogger.com (Site Confidence) — Tue, 07 Feb 2012 16:42:00 +0000

Since Google have now updated the look and feel of their analytics portal (including the addition of some lovely new features!) we felt it was time to update the instructions on how to go about filtering our synthetic visits from the GA portal.

Step 1) Open the Google Analytics account you wish to exclude the statistics from (note you’ll need to add a custom filter to each profile).

Step 2) Select the Options ‘cog’ in the top right.

Step 3) Select the Filters tab.

Step 4) Select New Filter.

Step 5) Then select a Custom Filter and choose to Exclude Visitor IP Address from the Filter Field.

In the Filter Pattern filed you’ll then need to add the Regex patterns which you can find within the monitoring portal (Support -> Service Information -> IP Ranges).

At this point you’ll need to split the regex string as GA will (still) only accept a maximum string length of 255 characters. The Regex expressions are separated with a “|” in the portal, so as long as the expression starts with a “^” and ends with a “$” you’ll be fine.

Save the profile and the rules will take effect straight away!

Q4 performance report

noreply@blogger.com (Site Confidence) — Tue, 24 Jan 2012 14:09:00 +0000

http://www.siteconfidence.com/media/news/article/12-01-10/Web_performance_is_for_life_not_just_for_Christmas.aspx

My main take-away from this article is that “Throughout 2011 retailers had failed to meet a target of 6 seconds download speed (as measured at 2 Mbps), but achieved it during Q4 with average speeds of 5.2 seconds. However the top ten e-tailers alone did far better, with average speeds of 4.33 seconds”.

So they achieved a performance gain, even though their page sizes are increasing (http://blog.siteconfidence.com/2012/01/page-sizes-are-increasing.html)!

Imagine what they can achieve if they started to decrease their page size and optimise…

Page sizes are increasing

noreply@blogger.com (Site Confidence) — Thu, 05 Jan 2012 16:12:00 +0000

A few weeks ago the BBC ran an article (http://www.bbc.co.uk/news/technology-16300000) using data from Steve Souder’s HTTP Archive which suggested that page sizes had increased over the last year. This seemed to be in line with our own observations. After all we download over 1.2 Terabytes of data a day from over 2.44 million tests.

I wanted to see if this was the case using our own data so I took the top 10 sites from the (current) Hitwise top 50 e-tailers over the last 12 months (2011-01-01 to 2012-01-01) and plotted the recorded daily average page size.

What did we see?

In general, page sizes did increase over the 12 months studied.

The normal patterns observed were either a continual and gradual increase over time or a sudden increase in page size as the Christmas rush started. For example, John Lewis maintained a home page of between 300KB and 400KB for most of the year but we then recorded the start of a dramatic increase to a high of 1.2MB over the following few days. Page size has since started to decrease although it is still c. 820KB (well above the yearly median).

The Apple home page also varied considerably, noticeably surrounding 2 significant product launches. On the 3^rd march the page size jumped from c. 665Kb to 980Kb for the launch of the iPad2, and then dropped back to 308Kb on the 28^thApril.

This was followed by another large increase (from 471Kb to 1.1Mb) on the 21^st July following the release of the Mac OS X Lion Operating System.

What does this show?

That web sites in the main are becoming larger. This could be due to more flash / JS based content, the BBC article specifically referenced that JS was largely responsible (source: http://www.httparchive.org/trends.php#bytesTotal&reqTotal).

This also shows that the average number of requests for JS objects has also increased since the start of last year. This could indicate that pages are getting more dynamic / interactive but also that more 3^rd party tags are being used (for things like user behaviour tracking, advertising and social ‘widgets’). The use of 3^rdparty tags needs to be closely monitored, given the SPOF (http://blog.patrickmeenan.com/2011/10/testing-for-frontend-spof.html) that they can introduce to pages.

Ultimately as we all know heavier pages will take longer to load (therefore reducing the end users experience) and cost site owners more to serve. Reducing page size where possible is going to improve any costs associated with the site (bandwidth, resource utilisation etc.) and will also help to ensure that users have a more pleasant browsing experience!

If you’re unsure how big your home page is right now, have a look at http://www.siteconfidence.com/test-your-site.aspx.

That’s a lot of throughput!

noreply@blogger.com (Site Confidence) — Fri, 02 Dec 2011 17:45:00 +0000

In the run up to Christmas we’re seeing lots of customers (old and new) running some pretty large tests to ensure that their platforms are ready for either a) the Christmas rush or b) their big New Year sales.

We’re in the process of running a project for one of these tests and hopefully in the coming weeks we’ll share some more specific details about it but for now I wanted to share the most significant headline number that we’ve seen so far.

This week we ran a test that managed to generate approximately 281,000 Page Impressions per Minute. That’s a fair amount of throughput! A quick (and dirty) extrapolation makes that nearly 17,000,000 Page Impressions per Hour!!

These were full page downloads, including all image / css and JS assets with full diagnostic capture (request / response header and html) on failed tests. Nothing was spared!

In total we downloaded c. 162 GB, which is a lot of content.

Not only was this incredibly useful for the customer, but it also served to highlight a few limitations within our platform. Some were quite minor (a few graphs weren’t properly displaying the large numbers correctly – a quick and cheap fix) and one was a little more expensive… we need to buy some more memory for our injectors! Both can (and shortly will) be overcome and the platform is ready for the next big(ger) test.

This was probably the biggest test that our new and improved load platform has seen so far, however I expect it to be beaten fairly shortly.

eBuyer is down - promotion proves too popular?

noreply@blogger.com (Site Confidence) — Mon, 28 Nov 2011 14:57:00 +0000

So the eBuyer site has crashed under load after their £1 gadget offer. Yawn.

As of 14:00 this afternoon the site is still down, due in no small part to this offer. You can bet however that the vast amounts of traffic didn't come as a surprise to the teams who had spent months organizing and then promoting the deals.

So do we put the blame on Ops again? They will probably be first up in front of the firing squad but I propose a new tack. All those who have a vested interest, and have been involved with the planning need to take some responsibility. Put the marketing team on the same performance and availability bonuses that the ops teams are on, that'd ensure that everyone was paying attention!

Ops teams should always carry out capacity planning exercises, that’s a given. If nothing else it’s an ITIL requirement.

But the business cannot just leave this with IT to fix. Its everyone problem if it goes wrong, as eBuyer are currently experiencing. Everyone involved needs to ensure that systems have been tested, ask for the reports and ensure that they prove that the application can support well in excess of the anticipated levels of throughput that are expected.

Performance testing is only as good as the data provided for the test but it’s also only as good as the goals that are set.

Was there a breakdown in communication between the marketing functions (who may have had an idea of the expected traffic levels) and Ops, who were perhaps just asked to carry out *some*capacity planning. Perhaps they then treated this as a box ticking exercise and didn't report the achieved metrics back to the business? If the business had known that they couldn't handle any more than X page impressions per hour / transactions per second would they have gone ahead with the promotion?

We shall probably never know.

The one thing that we can be sure of is that history will repeat itself again, and other retailers will do exactly the same thing.

I don’t want this to be a plug for performance testing, I want it to be a plug for common sense!! If you expect more visitors to your site than normal, test it. If you're making a change to something, test it. If you haven’t tested it for a while, test it! Something will have changed and you'll always get value from it.

A practical example of the benefits of compression

noreply@blogger.com (Site Confidence) — Wed, 09 Nov 2011 13:44:00 +0000

We all know that using GZIP compression can save bandwidth and improve performance of a website. But it’s interesting to see a real world example of the difference it makes. The graph you see here is the same website tested at exactly the same time by the Site Confidence monitoring agent. The only difference is that one monitor (in green) has compression of text-based objects enabled, whereas the red overlay does not. The difference is approximately 5 seconds, almost 50% slower!

Tested using the Site Confidence Monitoring Agent v8, at 512Kbps.

Manual vs Automated WPO

noreply@blogger.com (Site Confidence) — Wed, 14 Sep 2011 15:08:00 +0000

There is an on-going debate between those in favour of manual WPO and automated WPO. But surely lessons should be learned from both?

Automated WPO by using a server module such as mod_pagespeed is an easy, fast and economical way to "optimise" your site, but will only give you part of the solution. More advanced products like Aptimize or Strangeloop do a lot more, but also cost a lot more, they also take time to configure, massage and get working at the optimum level.

Cloud based services such as Yottaa can sit in the middle of this divide perhaps, offering an automated service, with no capital outlay that will give a general, automated optimisation layer. But by passing your traffic through their network, you are relying on another set of SLAs that need to be met, another "link in the chain" if you will.

Manually optimising your site can give many advantages, it means that developers can learn the "tricks" and should work that way by default in future, helping to ensure the site stays as fast as it can be. Some optimisations simply have to be done by hand, like domain sharding or setting up a cookieless domain. You can put browser specific optimisations in place based on your customer base, why put an IE6 optimisation in place if you know that only 1% of your visitors are using that browser? It just means that everyone has to download a bit of JS that 99% of visitors will never execute.

Of course there is a cost implication in manually optimising, and it is not insignificant, and certainly should not be underestimated. However, the cost should be looked at as a capital investment and one that will pay off by encouraging cleaner, leaner code from your developers, reducing bandwidth usage and infrastructure capacity.

There are arduous tasks that need to be done as part of an optimisation programme and they can lead to maintainability issues, such as minifying JS and CSS, or image spriting. Perhaps these kinds of optimisation are better be done automatically from maintainable code bases, and optimised by automated modules or serviced on the fly.

So there is perhaps an argument for using a combination of manual and automated techniques but relying on automated appliances or services to optimise your code, images and page layout exclusively can allow developers continue producing code that relies on someone else to optimise, giving way to a generation of developers producing bloated, inefficient code.

Another exercise in poor capacity planning?

noreply@blogger.com (Site Confidence) — Wed, 07 Sep 2011 13:18:00 +0000

A couple of weeks ago HP decided that they would be dropping their interest in WebOS devices, and with it they planned to get rid of a load of hardware (some of which they had only just launched) in a big fire sale. As you can imagine this generated a lot of interest in America (where the price drop was first announced) and both online and high street stores sold out of their available stock very quickly.

The UK quickly followed suit – The evening of the 22nd August saw the first few retailers begin their sale, with many others dropping their prices as soon as HP told them it was safe to do so.

What happened next was almost embarrassingly predictable – the majority of the retailers who dropped their prices (and some that hadn’t but still held a stock of Touchpads) saw their web sites fail under massive loads, as hordes of shoppers attempted to pick up a bargain. For the first few hours (while the web sites claimed that they had stock of the products) the sites were pretty unusable and any attempt to purchase items, whether related to the Touchpad or not, was bound to prove troublesome. I personally tried and failed many times, with most timeouts occurring at the basket steps which is when you least want your site to fall over (because in most cases you have logged in and reached the point of entering credit card details – a supposedly secure step which should be failure proof if only to give the customer peace of mind that their secure data is safe!).

But, do the retailers really care that much? They had a finite amount of stock to shift and presumably they sold it all in record time. They would probably have also captured some new accounts along the way, as new users tried in vain to pick up some Touchpads at knock down prices. The fact that other users would, for a period of time at least, had a poor experience may or may not have worried them too much.

This is similar to the phenomena of high profile ticket sales such as when Take That sold their tickets for their tour dates. In this example the artist announced that tickets would only be sold through a small selection of resellers. As soon as the tickets went on sale the websites of every single one of these resellers fell over – a state that they remained in for pretty much most of the day. A select few managed to pick themselves up for a few minutes, sell a few tickets and then fall back over again.

Again, did they care? As far as they were concerned, they had a finite number of tickets to sell and they knew that they would sell them all. If customers were looking to buy tickets on that day for a different artist they would probably have been very disappointed, but in all likelihood the resellers probably didn’t mind too much. They were going to sell a record number of tickets anyway, so could afford to lose a few additional sales. Is this the right attitude? From the consumers point of view definitely not – but from a business perspective (in the short term at least) it stacks up. Long term brand reputation is another issue.

Or is it? Are these instances unique events which attract a different breed of shopper? One that is less likely to be affected by problems like site outages? Ultimately, if my wife wants to buy tickets for Take That again she won’t care that the site was down last time, she’ll happily mash the F5 button all day until the tickets are safely ordered. I suspect that the retailers know this, and take the hit. Again that’s probably not the right attitude to take.

We’re constantly being reminded that the cloud is here to take away our capacity planning requirements. If you’re running out of resource, just spool up a new web / database / application layer right? Well, that assumes that the application has been written with scalability in mind. As we’re continually being reminded a move onto cloud based architecture isn’t a migration it’s a transformation, and it’s one that is slowly being adopted over here.

During the Take That sale Akamai were (anecdotally) pushing 3 Tbps! That’s an enormous amount of traffic, and something that most applications would struggle to cope with. One of our large retailers has a quite advanced queuing systems which sits in front of their web site on sale days. It welcomes the visitor, tells them that they are in a queue and tells them how long they have to wait before they will be let into the store. This achieves 2 very important things. 1) It allows the site owner to manage how many users are active on their application at any one time (so they won’t suffer any nasty surprises) and 2) it keeps the customer happy – they aren’t faced with an unresponsive site or nasty error messages.

I know that this retailer spends a lot of time testing the capacity of both their application and their queuing systems to ensure that both will function as expected when they are needed, and it seems like a very sensible approach to take. If you can’t scale your application on demand, you need to control how many people are using your application to ensure it remains within a controlled limit.

So, lessons learned? If you are a retailer and there is the possibility of large traffic spikes you absolutely *have* to plan for them. If nothing else, knowledge of the current maximum capacity of the application landscape is a must. Even if the resolution is to wheel in specialist hardware / systems for the big events then a dry run in a controlled environment is better than just crossing your fingers and hoping.

By the way, Site Confidence are about to launch a shiny new load testing platform, the first step of which is a portal refresh. If you’d like to take a look then drop us a line!

Ben

Written on my HP Touchpad

How to exclude page components using the monitoring portal

noreply@blogger.com (Site Confidence) — Wed, 22 Jun 2011 16:21:00 +0000

A Quick HowTo...

If you're monitoring a page but don't want the hits from our agents to request certain objects (tracking beacons, ad components etc.) then follow these instructions.

Step 1) Login and open the monitoring portal.

Step 2) Select Settings

Step 3) Select Monitor Settings -> Page

Step 4) Edit the page

Step 5) Select Show Extra Files

Step 6) Select Mode "Exclude (Partial Match)" and enter part of the string / URL that you wish to remove

Step 7) Save

All done!

Filtering Site Confidence monitoring stats using Google Analytics

noreply@blogger.com (Site Confidence) — Wed, 15 Jun 2011 16:05:00 +0000

Many of you use Google Analytics and we regularly get asked how to remove the Site Confidence monitoring tests from your results – particularly now that the latest Site Confidence agent is executing JavaScript by default.

I therefore thought it might be beneficial to provide a quick step-by-step guide, showing how quick and painless the task can be!

Step 1) Open Your Google Analytics portal and Edit the Profile on which you want to add the exclusion.

Step 2) Select Add Filter from Filters Applied to Profile.

Step 3) Select Custom filter.

At this point you’ll need to fill in a few pieces of information. Firstly, you’ll need to give your filter a name. Then select Visitor IP Address from the Filter Field drop-down.

Finally, you’ll need to enter the Filter Pattern. These Regular Expression (Regex) patterns can be retrieved through the monitoring portal (Support -> Service Information -> IP Ranges).

Note you will need to split the regex string as Google Analytics will only accept a maximum string length of 255 characters. The Regex expressions are separated with a “|” in the portal, so as long as the expression starts with a “^” and ends with a “$” you’ll be fine.

And that’s it! Save Changes and you’re all done.

Ben.

Discernible performance spike caused by mailshots

noreply@blogger.com (Site Confidence) — Thu, 31 Mar 2011 11:18:00 +0000

A quick observation for everyone.

A prospect who is currently on trial (no names, no pack-drill) noticed a significant spike in their recent performance.

The (diligent as ever) Account Manager dug a little deeper into this and found that the prospect had sent out a mail shot last night. Can you guess when this might have taken place?

This report overlays Visits Per Hour taken from the prospect Google Analytics account against the traditional Speed Analysis report.

The significant spike in response time is DIRECTLY related to the huge increase in traffic volume caused by the mail shot.

We can also observe that the traffic volumes created a huge impact on the Data Start time for the home page.

The Data Start times jumped from a baseline of near zero (good) to above 8 seconds at its worst (VERY bad). This time represents the time it takes for the first byte of information to be returned from the server to the client (browser). During this period users would have spent a lot of time looking at a blank browser… this will never help those all-important conversion ratios!

Anyway, thought you might be interested – back to report writing for me!

Ben

What should an SLA contain...

noreply@blogger.com (Site Confidence) — Tue, 29 Mar 2011 14:34:00 +0000

We were recently asked to help a customer define what an SLA should contain and thought it was worth sharing this with a wider audience.

DEFINE DOWNTIME:

Site unavailable – The site is telling the client it can’t get access or the client can’t find the site.

Taking too long – Decide a suitable download threshold at the point you believe a visitor might give up and leave. At this point the site might as well be down.

Lost functionality – This could be the homepage works fine but the login fails (and is needed to buy something for example). This is a little less clear cut.

Planned downtime – The argument should be that any type of downtime is downtime even when associated with maintenance. Some organisations makes a distinction between planned and unplanned downtime.

AGREE REASONABLE TARGETS:

Competitor analysis/benchmarking: See what the rest of the market aim for in terms of performance and availability. If 5 seconds at a particular testing speed is fast then what if all your competitors are at 4 seconds?

Customer expectations: What will your customers accept in terms of performance? If the site goes down will they come back? What if it goes down X times in a set period of time?

DEFINE DOWNTIME TESTING PARAMETERS:

Testing speed: If you are setting performance targets then they should be measured against a fixed and consistent testing speed.

Testing platform: If multiple servers are used to support testing then they should be the same and configured in the same way to ensure repeatability in testing.

Testing agent: Should perform in the same way as a real browser (like the SC agent) and should be consistent across servers.

If anyone thinks that we should add or amend anything then leave a comment!

London Web Performance Meetup

noreply@blogger.com (Site Confidence) — Thu, 06 Jan 2011 16:31:00 +0000

Just a reminder that the first London Web Performance Meetup will be taking place on Wed 19th January!

http://www.meetup.com/London-Web-Performance-Group/calendar/15729259/?from=list&offset=0

The guys from Seatwave have kindly agreed to present a case study on their performance improvement project for www.seatwave.com.

Perry & Ged have some great "real world" information on what worked for them from a technical perspective and how they managed the project overall.

Please RSVP on meetup.com if you are going to attend!

See you there from 6pm.

Site Confidence website has had a facelift!

noreply@blogger.com (Site Confidence) — Tue, 14 Dec 2010 15:24:00 +0000

The Site Confidence website (www.siteconfidence.com) has had a facelift today

The site has a new simplified site structure, information on our new products (such as Performance Analyser) and some great new whitepapers.

So head over to www.siteconfidence.com and let us know your feedback via the blog!

55 Killer Questions before you launch your new website!

noreply@blogger.com (Site Confidence) — Tue, 14 Dec 2010 15:02:00 +0000

What does it take to get ready for the launch of your new website?

What are the key things you need to do to ensure that you don’t join the ranks of Nokia, Apple, Nectar, the French and UK governments, and even Carla Bruni in having very public and high-profile disasters when you “go live”?

Launching a modern transactional website is a complex project, one that is increasingly “not just a IT problem” as you have to co-ordinate activity across multiple functional areas of the business e.g. Sales, Marketing, Customer Services etc.

Furthermore, delivery of many of these elements might be outsourced to 3^rd parties – designers, software houses, hosting providers, digital marketing specialists etc – that might be located in different countries, with different languages, cultures and in different time zones.

In the same way that a NASA Flight Director “goes around the room” to get a “Go / No Go” decision from each highly-trained specialist area before making the ultimate decision to launch a Shuttle mission the E-commerce Director has to try and co-ordinate these different areas prior to the website launch.

Unlike the NASA Flight Director however the E-Commerce Director might not have the luxury of years of experience, thousands of man hours of rehearsal, clear lines of communication, complete confidence in the technical skill of their staff nor in fact even have the ultimate authority to make that “Go / No Go” decision.
Wouldn’t it be great if someone gave you a list of things to check before “launch day” to ask each team to test if they are “All Systems Go?

The goal of this document is exactly that – an easy to understand checklist of the 55 “killer questions” things you need to do to ensure that your next website launch is a roaring success.

You can download a copy of the whitepaper here

The face of Velocity 2011…

noreply@blogger.com (Site Confidence) — Tue, 07 Dec 2010 15:04:00 +0000

The latest banner advert for Velocity 2011 caused much mirth here in SC Towers… because the gentleman in the blue shirt facing away from camera is none other than Site Confidence’s Mark Sweeting!

Mark is one of our performance technical consultants and came back from Velocity with lots of great performance optimisation tips & tricks.

The hunt is now on to identify who is the mystery woman he is talking to in the picture?

Does anyone know who the other face of Velocity 2011 is?

Performance Analyser is live – Site-wide cross-browser performance analysis.

noreply@blogger.com (Site Confidence) — Mon, 29 Nov 2010 14:02:00 +0000

Site Confidence’s new Performance Analyser tool has gone live today (and we are all pretty excited to finally launch it!).

Performance Analyser automates your website performance analysis by giving you detailed performance metrics on your web pages simultaneously in IE6/7/8 and Firefox 2/3/3.6!

What’s more, Performance Analyser can crawl your website, in the browser of your choice, to help you quickly identify the slowest loading, slowest rendering and just plain not working pages in your site.

So you can see the performance curve across your site and compare loading time to page size to look for anomalies (as you can see in the example below).

You can, of course, get the classic waterfall chart, complete with render start, DOM ready and other detailed timings at the page and object level.

If you want to know more, see a demo or have a trial please contact us!