Most businesses and organizations track site visitors, and they often use Google Analytics or a number of other third-party tools and apps to do this. But did you know you can track and analyze activity on your own terms, even for visits that happened long ago? How can this info help us leverage your website?
Websites live on servers, and servers keep logs of visitor activity for the purposes of maintenance and troubleshooting. In fact, most websites are “roomies” on a single server that could contain hundreds of sites. This means servers can fill up fast, and thus platforms vary in how long server logs are kept, but if your site is on a shared or private server, you could already have a treasure trove of info. Server logs can get large, so hosting companies have their own policies to keep things tidy, so it’s worthwhile to ask their tech support what their log retention policy is, and if they’d be willing to extend it for you. Ask them where the logs are kept, and how to reach them. Then remind yourself to go get them on a regular basis, even if you are not yet ready to use them so you the have long-term data you’ll need to spot patterns and trends.
“Access” or “visitor” logs are just ordinary text files, and the server will generate 1 file per month, and that file includes all of the activity, no matter what the source. In fact, you can open them in a spreadsheet like Excel without any special magic. Since each line in the log represents a file or request on a given page, a single page visit could generate dozens of log entries. These are server-level records, anonymous, and exist regardless of the site or web app’s programming. Cookie information and things filled out in forms are not in these logs.
If you are on a platform, you will probably not be able to directly download and view the raw log files. Instead, your platform may “digest” this information into a dashboard or tool to help you see patterns in visitor activity–which pages got visits, when was your site the busiest, which pages where the most popular, etc. But if you can get the raw logs, you can make actionable finds beyond what their dashboard provides.
No matter where your site lives, the logs contain the same info–they just might look different depending on the server. Each entry contains the IP address of the visitor, information about the browser, the date stamp (either UTC time your server’s timezone, not the visitor’s), and if the file was successfully fetched or if it was missing.
I can get deeper into the nuances of each fragment, but since this an overview, let’s focus on the value of these logs for a site owner.
- Missing files: Many CMS systems do not tell you if an image from your library is being used, so if someone deletes it, you may have a blog post from a year ago that is now missing that info. If your site refers to images, documents, or posts on other sites, they may no longer be around and you’d never know it. Turbine has done audits of logs to let clients know which pages and posts reference a particular image, document, or internal link. This helps ensure your site keeps looking professional with no broken or missing parts.
- Locality: When we access the internet from home, our internet provider (our “ISP”) provides us with an IP address. Our mobile phone providers do this as well, though they give us a new one every time you start an online session. This IP address associates you with a neighborhood, but does not map to your specific address or phone number. It’s called a “dynamic” IP address because it changes once in a while in accordance with the privacy rules (hence the value of cookies, which survive these changes) of the ISP or carrier. On the other hand, if you are at work, your business probably has a “static IP” when you use your computer or their wifi, and it can be tracked to a specific address (but not the individual visitor). If you’ve sent out an email campaign to hundreds of offices that includes a link to a specific landing page, you can not only count how many people clicked on the link, but where they were when they did so. If you’re a business that relies on local marketing, this is quite helpful. You may have subscriptions that charge you hundreds per month for this information.
- Return visitors: the average dashboard may tell you how many visitors returned to a particular page, but if you understand what the ratio is between new and returning visitors for a given page, you’ll have a better sense of how valuable your content is. A returning visitor is one who is getting closer to contacting you about your offerings.
- Breadcrumbs: knowing how many visitors saw a page is helpful, but where did they come from, and where did they go afterwards? Understanding the visitor’s “journey” can help you cater to them better with more accurate automated content suggestions than just showing them articles in the same category. Logs are anonymous, but you may still be able to discern commonalities among the content the visitor saw by looking for patterns from the user in relation to the timestamps of each entry, which would help you know your visitors better.
- Query strings: These are pieces of data that an be tacked onto a page link, and are often used to send someone to a specific product or landing page, pass along product, referral and tracking info, and more. Example: mypage.com/shopping?ProductId=464abc&campaign=summersale If you have an online catalog, your system probably gives each item an ID, and it’s likely in the query strings. If the link has a question mark, the query string is everything after it, and each name-value pair is separated by an ampersand (&). In the example, we see a productId and a campaign. Your system will see these values and show the visitor the product and the proper price. As you can see, this is a helpful way to see which products got a closer look, even if the visitor didn’t put anything in the cart and even if your cart is a widget from a third-party platform (like Shopify). If your software doesn’t notice these moments or encrypts them so they aren’t as legible as our example, you’re missing out out on valuable insights, especially in tandem with the breadcrumbs. Of course, Turbine would be glad to help give you that kind of leverage.
- Improved Visitor Tracking: If you think your dashboard’s visitor count seems a bit high, you may be right. Analytics often includes your own visits, and may even include you logging into your own admin panel. With smarter code in your reports, these can be accounted for.
This is not an exhaustive list, but as you can see, your visitor logs are a great resource that won’t cost you anything to obtain. If your current platform or tools don’t give you a way to leverage them, or you’d like to kick around some ideas about how your business could leverage your logs, please feel free to contact us.