Image © Juja Schneider

When discussing possible load test setups with our clients, we usually need to refer to these key terms: visits, sessions, requests, hits, page impressions, and page views. Actually, we don’t need to discuss all of them, but some are occasionally brought up by the customer, some are requested by us, depending on the context (and complex enough to be discussed in a separate article).

Many clients of ours have told us that it’s sometimes hard not to confuse these terms as they seem to denote the exact same thing. Today’s article is thus meant to give you an overview on their definition and help you distinguish them. Here it comes.

Visits

Basically, a visit occurs when you send a request to a server and, as a response, the website you requested is displayed. The display of the page is what we call the page view (which is covered in more detail below). Take, for instance, www.amazon.com: you enter the URL, that is you send a request to that server. As a result, you arrive on the shop’s homepage. In doing so, you generate a visit and the website owner now knows that someone visited their site. Accordingly, a visit consists of at least one page view. Typically, though, you’ll decide to browse through the shop, thus producing further page views with every single one of them adding up to your visit regardless of what you actually do while staying on the site.

Your visit ends when you become inactive for whatever reason, for example, when you stop clicking links or when you overall close your browser. The server then deletes or deactivates the data that has been collected during your visit. Depending on the web application, the time period before this actually happens varies from 30 minutes up to 24 hours (see below for more details). Compared to the real world, a virtual visit is therefore not much different from a visit in an actual store: regardless of how you spend your time in there, the store owner will consider you a visitor and they will eventually forget about you when you decide to leave their store for a longer period of time.

A visit defines three metrics that are important for us to know: the visit duration, the number of page views per unit of time, and the time period between two page views (thinktime). By the way, the Internet is crowded by an insane number of machines that also generate visits and that cause statistics to get messed up. There are a couple of technical measures trying to filter out these irrelevant visits. Let me know if you’re interested in how that can be accomplished and I’ll talk about it in another article.

Page view

As mentioned above already, a page view, or page impression, is the display of a website you trigger by sending a request to a server. It was not long ago that one request was sufficient for the page you requested to be displayed; via further requests, the browser then added images, CSS, and Javascript. In the modern web, the page view term is used in broader contexts as AJAX and recent user concepts don’t exchange complete websites but only modify small parts anymore. Thus, a page view is actually an interaction or action most of the time.

But let’s stick with the old-fashioned page view for now. It gets initiated by a request and terminated by the response from the server. This sequence of events is also referred to as hit. A site may contain other elements, such as images and CSS. Each of these elements is again processed by a request/response sequence and leads to another hit. Thus, a page view is made up of an HTML component and many embedded elements with each element and the HTML component causing further hits. Important metrics are, for instance: page load time, page size, and view time, namely the time period until the next click.

Hit is often the business term for a request that, in turn, is the technical term for a single data transfer.

To sum this up: A visit consists of one or more page views and has a certain duration while a page view has a runtime and consists of elements referred to as request or hit.

Sessions

Now, how does a session differ from a visit? In simple terms, a session is the technical picture behind a visit. The software you take advantage of while browsing through a webshop has to remember which requests belong together so that functionalities such as a login or a cart can actually do the things they’re supposed to do.

Sessions consist of data that summarize certain information concerning your visit which is why they are also called session information. Usually, these information have a limited lifetime as they are subject to a session-timeout. In most cases, this time-out amounts to something between 30 minutes to 2 hours. As soon as your visit ends, time is ticking down and, upon reaching the session-timeout, all data is deleted. If you continue your visit before the time has elapsed, the countdown resets to 0. To illustrate this with our real-world example, imagine you decided to buy something in the store you walked into (visited) but at the register you suddenly realize you didn’t bring any money. You can leave your cart at the register for a while to go get some cash but if you don’t return in time, the cashier will assume you’re not coming back and empty your cart.

Technically speaking, the number of sessions equals the number of visits. Due to business-related aspects, however, visits are counted differently than sessions so that the number of visits is usually lower than the number of sessions per unit of time.

Summary

In a few words:

• Visits mostly equal sessions, where session is the technical term and visit is the business term. A visit consists of at least on page view.
• Hits and requests are the same, request is the technical, hit is the business term.
• A page view is a single complete page delivered. At least one request is needed to serve it, mostly many requests are fired to assemble a page. Nowadays, page views tend to be interactions because often full pages aren’t loaded anymore; instead, only pieces are dynamically changed (the famous Ajax magic).

Confused? Feel free to ask!

Usually, we just  measure the performance of our customer’s applications and talk about it, but from time to time we have to set an example ourselves.

In the last couple of weeks, we increasingly felt that our blog isn’t loading fast enough to deliver a satisfying experience. You know that when you can feel it, it might be too late already. Additionally, SEO is about content and performance and our blog is an important marketing tool for us.

That’s why we went on a quest for improving the performance of our company’s WordPress-based blog. Our motto: “Don’t just complain about the lack of performance, do something about it!”

Step 0 – Measuring

Measuring is believing and so we started with this WebPageTest result. As you can see, the initial performance is bad, a lot of content is not properly cached, and rendering started after 2.6 sec. Time for some serious tuning.

Step 1 – Reading

Tuning requires you to know what to tune. Thus, we read the famous Best Practises for Speeding Up Your Web Site by Yahoo. A similar article by Google can be found here. If you deal with web site performance in any way, you should read this. We consider it mandatory for performance and web testers.

Step 2 – Wordfence

One of the first things we did was disabling Wordfence because we had the strange feeling that the performance degradation of the blog had started just recently – a couple of weeks ago, we’d installed Wordfence in response to the WordPress login probing attacks. Yes, the disabling made us faster: first byte time came down from 1.7 sec to 0.7 sec. This was awesome… but who wants to run WordPress without protection? So we tuned Wordfence and disabled the firewall and live traffic detection, basically just keeping the login security and scanning features.

Step 3 – JavaScript

So, what was next? We realized that Slimbox had an impact because of the MooTools library it used. Not really a huge impact, but the JavaScript held back the rendering and further loading. We picked wp-jquery-lightbox as a replacement which didn’t change that much but got a little faster; Jquery seems to be a little better optimized in terms of JavaScript performance and size.

We also updated the SyntaxHighlighter since we’d been using an older version that was no longer under active development. Yet no relevant change in performance here.

Step 4 – Thinking

That’s basically all you can do, provided that your content is okay (properly compressed images) and that you have a fairly small CSS file for your theme. Because changing the hoster, tuning the database, or even replacing the webserver is a little overkill for a low-traffic blog, we could only take the caching route next. You can go back to the Yahoo and Google article and check what is possible without starting to reinvent the wheel aka writing a new blog, changing the theme, rewriting all content…

Step 5a – WordPress Caching

When you look up tuning tips for WordPress, you immediately find W3 Total Cache and WP Super Cache. Based on our reading of reviews, Super Cache seemed to be a little easier to handle, so we tried that. Somehow we failed though: it didn’t do anything and the performance stayed the same. As it’s not that easy to use, we switched to W3 Total Cache.

Of course, Total Cache is overwhelming as well and it took a while for us to realize that it couldn’t save its settings due to the file system permission handling of our hoster. Thus, we updated the .htaccess files manually. Since we also pull content from our Piwik installation, we added some of the .htaccess settings there as well. These settings basically refer to  serving the content as cacheable to the browser, avoiding refetching as well as delivering the content compressed. CSS, JS, and HTML can be compressed quite well and so it’s recommended to run mod_deflate for reducing bandwidth consumption.

Step 5b – W3 Total Cache

We had to go through a lot of trial and error to get the performance where we wanted it. Total Cache offers so many options that we had to try over and over again. Don’t forget to measure the  settings several times since also the measurement tool can have a bad minute as well! Also, if you run on a shared hosting environment, you’re not alone and other traffic might influence your measurements.

Step 6 – Measuring Again

After all this was done, we measured the final performance. The first bytes now came in after 0.22 sec and rendering started after 0.7 sec. Wow! We’d begun with 1.7 and 2.6 sec!

Step 7 – Enjoying

This basically means that we improved the performance of our blog by factor 4 to 8! Google and Bing, feel invited to drop by and index our content! We hope that our visitors enjoy the improved performance as much as we do. If we wanted to further enhance it, we had to move to a better server that can be tuned by dealing with database and network settings on a lower level. However, this doesn’t make any sense right now, micro-tuning doesn’t do any good here.

Always keep in mind: you can only test what you know and understand. It is this knowledge that allows you to evaluate client performance the next time you have to test it, and it also provides useful feedback on how a problem can be solved or in which direction you have to look for a solution.

We’re looking forward to hearing about your client site performance tuning and measuring experiences. Please share your WordPress tuning and operational procedures as well. Happy performance blogging!

The upcoming Xceptance LoadTest 4.2 will discontinue the direct support of JRuby as a programming language for writing test cases. This will not change the way Java and JRuby interact and therefore you can still use all Java components and integrate them into your JRuby-based test suite. You can also still write test cases in Ruby and use XLT-APIs.

You can read about this feature, in case you are not sure whether or not you will be affected by this change: XLT 3.3.0 JRuby support release note.

Version 4.2 of Xceptance LoadTest will not continue the JRuby package anymore (jruby.jar). The test execution scripts for JRuby-based tests and all demo test cases will be removed as well.

If you have any questions regarding this change, please do not hesitate to contact our support team directly.

XLT 4.1.8 has been released. It contains a couple of improvements and bug fixes. You can get the latest version here: https://lab.xceptance.de/releases/xlt/4.1.8/.

These are the two most important changes.

Optimized scanning CSS rules

XLT simulates the download behavior of a browser as close as possible. With com.xceptance.xlt.css.download.images set to onDemand, XLT carefully considers the download of image resources referred to in CSS files. It checks all CSS rules if they apply to all elements in the DOM tree and, if so, extracts the resource information for later download. This process was previously very expensive and ran for up to several seconds when a page was complex and a lot of CSS rules had to be matched. This has been optimized heavily.

Configurable retry behavior for keep-alive connections

If persistent connections (keep-alive) are enabled, test cases might failed sporadically with an exception such as:
java.lang.RuntimeException: org.apache.http.NoHttpResponseException: The target server failed to respond

This exception occurs only when the request was sent while the connection was about to be closed on the server side at the same time.

The underlying HttpClient retries requests if the connection was closed by the server. However, by default it retries only idempotent operations such as GET and HEAD, but not POST and PUT. In order to avoid these connection errors, the user can now configure whether non-idempotent operations are to be retried as well. Common browsers seem to use the same behavior and most certainly assume that the server did not start to process the request when it closes the connection immediately without responding with a proper HTTP status code.

A new boolean property com.xceptance.xlt.http.retry.nonIdempotentRequests in config/default.properties controls whether or not operations such as PUT and POST are retried in case of certain networking problems. Note that idempotent operations are always retried.

We just released Xceptance LoadTest 4.0.5. It is a minor update and recommended for everyone. But you might have special interests if you use the Script Developer heavily.

Besides a few defect fixes, release 4.0.5 delivers four great improvements to speed up test case creation and maintenance with Script Developer and make your work more productive.

• The XLT Script Developer runs on Firefox 4 and 3 now.
• Test variables are now resolved recursively, so you can use variables within resolved content.
• There is no need to open modules anymore if you want to edit a line or two of it. Also enabling/disabling of module code can be done easily from the main test case. This saves time and aids script maintenance.
• During script debugging and script execution, you can now evaluate assertions instantly to see whether or not your verification expression will match.

The full set of release notes can be viewed directly in the release area. You will also find documentation and the download link there. As usual, th

Today we released update 4 of XLT 4.0. You will find all the details about it at the usual place: https://lab.xceptance.de/releases/xlt/4.0.4/. Check out the release notes too. This release addresses minor script developer defects.

As always, updates are free for all customers and all people using XLT despite the license.

There are new public Amazon EC2 images (AMI) available each running 4 agentcontrollers of XLT 4.0.0.r6019 on ports 8500 to 8503. Feel free to use them for your load testing purposes.

• EU-West: ami-9b5064ef
• US-East: ami-5647b63f
• US-West: ami-539dcd16

The usage of the images is free or charge but you have to pay your AWS usage costs of course. Please keep in mind that you need a valid license to run a load test with more than 5 virtual users.

We just released Xceptance LoadTest 4.0. This release of our load test software got some really nice feature enhancements to make your regression testing easier. So we stick to our general software approach: One tool for regression and load testing. One set of scripts for both purposes.

Script Developer

As an alternative to writing test cases in Java, you can now use the XLT Script Developer to create script test cases. Script test cases are based on a simple syntax and a reduced set of operations, which makes them a perfect fit for non-programmers. Only the Script Developer, which is an extension to Firefox, is necessary to create, edit, and manage basic script test cases.

To create a new script test case, the test designer simply uses the application under test. All interactions with the application are recorded in the background and stored to an XML script file as a sequence of script commands. While recording, assertion commands to validate the web pages may be inserted manually. From the Script Developer, script test cases can be replayed in Firefox at any time to quickly check whether the test case still runs successfully.

Existing script test cases can be modified later on, for example, to add new or delete obsolete commands. Common command sequences, which could be reused in other test cases as well, can be refactored to parameterizable script modules. Finally, any recorded value can be extracted out of the script into a test data file to separate test data from script code.

Script files can also be run outside of the browser, via the XLT framework, which simulates a head-less browser. This mode is suitable for unattended test case execution, during functional or load tests. When saving scripts, the Script Developer also creates JUnit test case classes as “wrappers” around script test cases, which serve as a bridge between the XLT framework and the script world. This way, from the framework’s point of view, script test cases are in no way different from test cases written in Java.

More Data to Query

For improved tests accuracy, you can now query the request and response data and run assertions on it. This permits checks on the communication because not all requests are reflected in the DOM tree.

Improved EC2 Handling

AWS (Amazon Web Services) added the ability to tag EC2 resources to simplify the administration of your cloud infrastructure. As a form of meta data, tags can be used to create user-friendly names and improve coordination between multiple users. The XLT EC2 administration tool ec2_admin features an additional menu which lets you select your EC2 resources based on the tag name.

Better Automation

To improve automation of tests, we added the ability to pass properties on the mastercontroller command line. Additionally the test definition file for the test suite can be redefined on the command line as well.

Faster Work Flow

When test goes wrong or a logging is turn up, the data to download from all agents can be pretty big. To get a fast or selective result, you can now decide how much data you want to download.

JDK Compatibility

Beginning with v4.0, XLT requires a Java virtual machine 6 or above to run. Java 5 is not supported any longer. The reason is the end-of-life announcement for JDK 5.

Misc

We refreshed HtmlUnit and updated it to version 2.8, Ruby got updated to 1.5.1, and WebDriver is now v2.0a6. The event API got simplified and is now easier to use.

Where to get it

More information about the release, the quick start guide, and the manual can be found in the release area. Of course, the full download of XLT 4.0 is available there too. 

We are looking forward to your feedback, comments, and of course… Happy testing!

People asked for a rough guide for running load tests against their live site and whether this is a good idea at all.

Well, let me first say that test environments exist for exactly that purpose. So if you already have something live, of course you used it before going live for load testing, and now you cannot run another test there… yes, you need a test environment. This is well spent money for several reasons:

• Created data will not pollute your installation aka log entries, analytics data, orders in the database, created users, and so on.
• You do not have to take your live site down for testing.
• Problems during testing will not leave your live site down or take it down.
• Your hosting company or IT department might charge for all test traffic against the live site as it would have been real traffic (revenue share, bandwidth utilization), so having your own testing realm makes expenses more predictable.
• You can easily change code and retest when changes are necessary. You can profile, you can debug.
• You do not risk problems with traffic going to live systems, such as payment due to configuration or testing errors. If you already had items from test orders piling up in your office, you know what I mean.

So, get yourself a test environment. Spend the money and enjoy the freedom to measure, debug, and tune.

P.S. Of course, there are situations, where you cannot replicate a live environment or you cannot find application problems in a test setup due to totally different timing behavior… Well, this indicates only that your live environment is already screwed.

Heute mal wieder etwas aus der Reihe “Erfolgreiche Fehler” oder “Unsinnige Dialoge”. Gefunden im Nautilus von Ubuntu 9.10.