Alternative Mobile Versions FAQ

How To Optimize Mobile-Only Websites

A few years ago, mobile devices were just used for phone calls. But nowadays, the traffic that our websites receive from smartphones just keeps increasing to a level that their importance cannot be denied anymore. And with new technologies and opportunities come new challenges and questions.

To make things easier, we are sharing here a list of questions and answers about how to optimize mobile-only versions of your websites, taking into account technical, user experience and SEO considerations.

• Why would redirecting to a mobile version make sense?
• Should you display different content for mobile browsers in the same URL, or redirect them to a different, mobile-only URL?
• Should you create different versions for feature phones and smartphones?
• How is a mobile URL marked as a mobile, alternative version?
• If there is no corresponding original page, where should a canonical tag inside the mobile version point to?
• How can you detect mobile browsers to serve them different contents?
• Should Google's mobile crawler be considered in mobile browser redirection scripts?
• Server-side redirection or Javascript redirection?
• Permanent 301 redirect or temporary 302 redirect?
• Should you include a vary HTTP header in mobile URLs?
• Should you include a link to the original, regular version in your mobile-optimized version?
• Where should a link from the mobile version to the regular version exactly point?
• How to allow mobile users browse your regular version?
• What about redirecting desktop / regular users accessing your mobile version to your regular version?
• Are tablet browsers mobile browsers too?

Why would redirecting to a mobile version make sense?

Responsive web design presents several advantages when compared to creating a detached, alternative version just for your mobile users:

• Maintenance can be simplified. Even when responsive websites can increase the level of complexity, only one website would have to be modified. Content-wise, all your contents would remain on just one place, making it easier to expand and update your site for both, regular users and mobile users.
• SEO can be simplified. Presenting a single version of your website for all your users with (mostly) the same content avoids the hassle of notifying search engines about both versions, making clear which one is the main version and which one is the mobile-optimized version.

However, even when responsive web design is probably the only trend that will survive in the not-so-distant future, creating a responsive web design instead of a mobile version may not be the best option under certain circumstances:

• Budget constraints. Responsive designs require additional planning in advance and testing in a broad array of devices and screen resolutions, for all the kinds of content.
• Outdated phones. If most of your users still rely on feature phones or old smartphones, but still do want to navigate a mobile version of your site, then you better make that version very lightweight and fast, specifically designed with simplicity and speed in mind.
• Legacy code. If you already do have a mobile version in your website, or a website that would be very hard to turn into a responsive website, then working with your existing versions would make sense.
• Context restrictions. The goals of your users while on the go may differ from the goals of other users accessing your website from a desktop computer. There are some additional interface and context restrictions as well: typing on a small touchscreen while walking down the street may render your web forms useless; reading on a tiny mobile screen may render your detailed, long pages overwhelming.

If you are under one of these previous scenarios, you will definitely want to keep using your separate mobile website version, and optimize it as much as possible so:

• You will be serving easy-to-navigate content to your users.
• It will be easy to discover for search engines.
• Your mobile version won't compete with your original version, cannibalizing your search engine results.

Here is a list of frequently asked questions about how to optimize the performance of a standalone mobile version.

Should you display different content for mobile browsers in the same URL, or redirect them to a different, mobile-only URL?

There are two approaches to serve mobile optimized content to your site users: keeping them in the original URL but sending them different, mobile optimized content, or redirecting them to a different URL where they would find mobile optimized contents.

The first approach has a SEO-related advantage: all links pointing to one of your mobile URLs will also pass their ranking to the main URLs, since both are the same.

However, this approach also has bigger risks as well. Issues with this configuration, like failing to tell apart your mobile users appropriately, may lead to search engines think that you are sending shallow content to all users, or creating a URL masking scheme.

There could be issues with your mobile users as well. Without the URL providing additional feedback about the version they are browsing, your users might be confused about thinking this is the only version of your website (if your "go to original version" link weren't very visible.)

That's why redirecting to a mobile-only URL still makes sense, as it can be technically easier to work with, pretty clear to understand, and very well organized. But you need to make sure such an URL is marked as just for mobile browsers, as an alternative version, so it won't compete with your regular URLs in search engine rankings.

Should you create different versions for feature phones and smartphones?

While smartphones boast not-so-tiny touchscreens and powerful processors, those old mobile phones out there, the so called feature phones, will lack enough screen real estate or processing power to deliver a good browsing experience in most websites.

The answer to this question depends on your business. Your browsing stats will be priceless for this purpose. Most stats tracking software will give you a detailed breakdown about the hardware that your visitors are using to navigate your site. If that list contains a decent percentage of old mobile phone users, then creating a version optimized for featured phones would make sense.

However, mobile phones are quickly evolving. It depends on your target market and on the country where your business is located, but chances are that feature phones will quickly disappear in favor of more powerful smartphones. And taking into account that the processing power of smartphones is coming closer to being little desktop computers, chances are that your future website design plans should consider only smartphone-optimized versions.

At the end, this is what the other big web design trend of responsive design is already hinting: in a not-so-distant future, desktop versions and mobile phone versions would only be distinguished by the layout used to present information. In the meantime, a single alternative fast-loading and concise mobile version should suit the needs of all your mobile users.

How is a mobile URL marked as a mobile, alternative version?

Google recommends tagging pages with alternative versions following this approach:

• The regular, original page should signal that there is a mobile-friendly version serving similar content.
• The mobile-optimized versions should point at their related regular pages, marking them as the main, canonical pages.

Screen size is used to discriminate mobile phones, while the media type handheld is used to target feature phones. In the most common scenario where there's just a mobile-friendly version, these are the alternate link tags that you would include in your main, regular version:

 <link rel="alternate" media="only screen and (max-width: 640px)" href="http://www.mobileURLHere" />

<link rel="alternate" media="handheld" href="http://www.mobileURLHere" />

And this is the canonical link that you would include in the mobile version:

<link rel="canonical" href="http://www.mainURLHere" />

You can also tag mobile versions in your sitemap.xml as alternative versions. Each alternative mobile version should be associated to its related regular version, so the sitemap of your website would look like this:

<?xml version="1.0" encoding="UTF-8"?>

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"

  xmlns:xhtml="http://www.w3.org/1999/xhtml">

<url>

<loc>http://www.mainURLHere</loc>

<xhtml:link

    rel="alternate"

    media="only screen and (max-width: 640px)"

    href="http://www.mobileURLHere" />

</url>

</urlset>

It is important to make sure you are redirecting exactly to the URL specified in these alternate tags, or you could be sending confusing information to the search engines, and you could even risk getting your website flagged as spam.

If there is no corresponding original page, where should a canonical tag inside the mobile version point to?

If your mobile site doesn't mirror your original site in full (that is, having a mobile-optimized page per each original page) then the quickest approach would also be the safest: small mobile websites can redirect all their pages to your main url in your original version, ensuring all of them pass pagerank to your main webpage.

Take into account that, in an ideal case scenario, every single page of your main website should be having a mobile-optimized page. Other scenarios can lead to bookmarks that don't work as expected across different devices, or, as Google calls them, irrelevant redirects – unless your mobile version is specifically crafted to serve an optimum version considering your mobile users' context and market niche.

Update: in this official blog post titled Changes in rankings of smartphone search results Google updated what they consider an irrelevant redirection. According to this report, mobile-accessed pages without a mobile version available shouldn't redirect to your main mobile page, as that would be considered an irrelevant redirection. The recommended approach for pages without mobile versions is to lead the user to the original, non-mobile-optimized page.

Other than that, the recommended suboptimal approach according to Google is having a mobile-optimized page for every single of your regular pages (thus removing the risk of irrelevant redirections.) And the optimum recommended scenario would follow a fully responsive website approach, with a single version that would work nicely on all devices.

How can you detect mobile browsers to serve them different contents?

Mobile browsers identify themselves through their own user agents. By identifying their user agent you can send them different contents or redirect them to a new page.

You don't need to create a whole list of user agents by yourself. There are some scripts out there that would take care of that for you. Detectmobilebrowsers.com is an open-source mobile browser detection library that will do the trick, offering scripts in several different programming languages (Apache server scripts, Javascript, ASP, PHP, etc.)

Be careful with this tactic, though, since new mobile devices, not considered in this list, might eventually appear. There's a slim chance of your mobile detection script running outdated and classifying new devices into the wrong category. So make sure you keep updated detection rules as well.

Should Google's mobile crawler be considered in mobile browser redirection scripts?

Google has a Googlebot-mobile webcrawler, specifically dedicated to crawling mobile versions, which is completely independent from their regular Googlebot. So it would make sense that this crawler reached the mobile version of your website.

However, Googlebot-mobile presents itself using the user agent of regular, well-known smartphones. You don't need to discriminate Googlebot-mobile in your browser-detection scripts. Google warns that doing so can be flagged as URL cloaking, which would bring you a penalty from Google.

Server-side redirection or Javascript redirection?

There are a couple of different approaches when redirecting your mobile users to a different mobile version. You can perform the appropriate user agent checks on your visitor's web browser by using a Javascript redirection. Or you can go ahead with a server-side redirection, so your web server performs the browser agent checks (either directly on the server engine by using .htaccess rules, or in a server-side programming language, such as .NET, PHP, Ruby and so on.)

The problem with a Javascript redirection is that part of your original page needs to load before your mobile user is redirected to the appropriate version. On the other hand, a server-side redirection would leave the user agent checks and processing to the server itself, preventing your user from loading unnecessary content.

Taking into account that mobile users have limited bandwidth and processing power, server-side redirects are the preferred approach since they will be faster.

Permanent 301 redirect or temporary 302 redirect?

There are two main different types of server-side redirects that are triggered by specific HTTP response headers directly sent from the server:

• A 301 redirection is a permanent redirection, indicating that the original URL doesn't matter anymore, so all queries should be redirected to the new URL.
• A 302 redirection is a temporary redirection, just signalling that visits to the original URL are redirected to a new URL at the present time, but that this is a temporary situation expected to be reverted.

If you use response codes to redirect your mobile users to the appropriate, mobile version, in Google's own words:

For this purpose, it does not matter if the server redirects with an HTTP 301 or a 302 status code.

So it really doesn't matter what kind of redirection you are using to send your users to your mobile version.

However, let's consider a worst case scenario: if some misconfiguration or glitch in the process is preventing both versions of your site from being understood the right way from a search engine's perspective, using a permanent 301 redirect from your regular version to your mobile version would send most or all your pagerank, traffic or users to your alternative, simpler, mobile-optimized version. Such a scenario would be a disaster for your SEO strategy.

That's why using a 302 temporary redirect pointing towards your mobile version is a better strategy in terms of risk prevention.

Should you include a vary HTTP header in mobile URLs?

Google mentions in its guidelines that, in case you use any kind of redirect, any redirecting webpage should also be outputting a Vary HTTP header to notify potential search engine crawlers that mobile crawling is also in order, since the page can lead to different results depending on the user agent used to browse it. This way, every mobile page would be crawled, and the structure of your site would be better understood.

This also holds true for websites using the same URL to serve optimized content for both, mobile users and regular users (dynamic serving.) In this case it can become even more important, as there wouldn't be any other way to tell the search engine spiders that a mobile version exists for the present page. You would also make sure that the mobile optimized version wouldn't be considered as masked content or as the main version related to this URL, since you would be announcing its existence through this header.

The Vary header should be sent indicating that the contents in that page can vary depending on the user-agent, so it should look this way:

Vary: User-Agent

Should you include a link to the original, regular version in your mobile-optimized version?

Definitely. Some users will still want to access the full contents of their main version if they feel that they are missing part of the content, or if they just have a smartphone powerful enough for regular browsing.

If your mobile-optimized version is simple and short enough (as it should be,) the link to the original article / normal version could be safely included at the bottom of the page: there wouldn't be any need to go to the regular version if the mobile version provided enough information by itself. Going to a normal version is rarely a decision taken in the first second.

Make sure to display this link to your main version in a very visible way, though. You really do want to make your users know there's still a different, potentially more complete and eye-catching version that they could visit if they wanted to. This becomes specially important if you are serving your mobile contents from the same URL, as the URL wouldn't offer any additional clue about the existence of a regular version.

Where should a link from the mobile version to the regular version exactly point?

In the ideal case of having a mobile website composed of a mobile page per each regular page, the links pointing to the regular versions from the mobile pages should precisely target their corresponding regular page, talking about the same subject and offering similar content.

However, things can be a little bit more complicated if your mobile website comprises just a subset of your whole regular site, which can still make sense to offer a simplified, more straightforward version of your contents to the specific market niche composed of your mobile visitors.

If there were no matching page, offering a link to the homepage of your website from its mobile version could seem like a good idea, since the homepage presents the most easy-to-access gateways to all your content.

But that's not enough. Consider the following scenario: a user performs a specific search from his web phone, clicks a specific search result, and is then redirected to the homepage of your mobile version instead. This can be okay as long as your user appreciates the ease of navigation of your mobile version, and as long as he finds relevant information. But, what if he just wants the answer to his original, specific query, and thinks that his smartphone should be able to handle browsing the full, regular webpage? Giving him a link to your main homepage will feel like a burden.

That is why you should also offer a link to the page your mobile users were trying to reach before being redirected to the mobile version of your website. You can annotate the page where a redirection was started (for example, in a parameter, cookie or session variable.) Mobile users pleased by the ease of use of your mobile version would remain there with no problem. And mobile users wanting to reach the original contents that triggered their click in the search engine results would still have the option to view the original contents they were originally thinking about.

How to allow mobile users browse your regular version?

Mobile users should be redirected towards your mobile-optimized version. So when one of your mobile users prefers trying to navigate your full website using a powerful smartphone, and clicks a link to your regular version, how to prevent him being redirected back to the mobile version, and stuck in an endless loop?

One of the most straightforward approaches to solve this issue is to add an additional parameter to override this redirection for users who explicitly request to see full versions from their mobile phones. Just a query parameter appended to the target URL can be enough. The target, regular webpage would check for the existence of this parameter. If found, any redirection would be skipped this time, allowing your mobile user to navigate the regular version of your website.

Nevertheless, that might not be just enough. That same user could still want to keep navigating other regular sections of your website. If such parameter weren't kept, your user could be redirected back to a mobile version when clicking a new link.

So once a regular version is requested by including a parameter to override mobile browsing, this preference should be stored in a cookie for your user, or even just in a session variable. So your user won't have to worry about being redirected to your mobile version again.

If a cookie or session variable were set for that user, no regular page would redirect such specific mobile user.

On the other hand, this user can still go back anytime to your mobile version, not by an automated redirect, but by pressing his “back” button on the web browser of his mobile phone. This is a rare scenario that might only happen if your user decides that the regular version is too much for his phone after explicitly requesting it, but being covered about this case is still worth for optimum user experience purposes.

What about redirecting desktop / regular users accessing your mobile version to your regular version?

This is what Google calls bidirectional redirects. It prevents your desktop users from landing on a mobile-optimized version, which in their large screens would look rather limited or empty, and thus, wouldn't create the appropriate first impression.

If you have an updated mobile agent detection, then bidirectional redirects can make sense. It is very strange that a desktop user would rather see the incomplete or limited mobile version instead of the full, original version of your website.

Independently from that redirection for desktop users, some mobile phone users would still like to access your regular pages. Having a link offering to visit the normal version instead would come here into play.

The only catch about bidirectional redirects is that you need to make sure your code is working perfectly. If not, you could leave your users stuck in the wrong site version or in a redirection loop.

Are tablet browsers mobile browsers too?

Tablets have some points in common with mobile browsers: they provide their users with some mobility, not-so-big touchscreens, and less processing power than desktop computers.

However, most modern tablets already allow regular navigation in regular webpages with ease. Their processing power is enough to move normal webpages, and their screens (even the 7-inch ones) are big enough to provide good website readability.

That's why I suggest not treating tablets as mobile devices, and to exclude them from your user-agent detection scripts.

Besides, it is still possible to redirect tablet users to your mobile version as well. This can make sense if your regular version is not so tablet-friendly (if it takes too long to load, if it features Flash content that won't work in iPads, or if it just targets a niche where you wouldn't include your tablet users.)

If you are using the user-agent-detection script previously mentioned, detectmobilebrowsers.com, you would just need to add the following code to the first regular expression to treat iPads, Kindle Fire, Playbooks and Android tablets as mobile devices as well:

|android|ipad|playbook|silk

0 comments

Permalink

Posted by ESV

Best 5 ways to use Google Analytics notes

What are Google Analytics notes?

Google Analytics has a new feature that allows you to create notes for your statistics. With Google Analytics notes you can attach a description or a comment to any date in your graph. It is just a matter of clicking a point in your graph, and then, writing down your annotations.

Why are Google Analytics notes so important?

This feature is very interesting because, till now, you had to keep a changelog for any maintained sites. But by embedding notes straight from Google Analytics, attached to specific dates, you can keep your historic of changes together with your statistics.

In a nutshell: with Google Analytics notes you better understand how is your site behaving. And only by understanding your measures you could optimize your site.

The best 5 ways to use Google Analytics notes

Since you can write your annotations in Google analytics to understand perfectly how it works (and being thus able to improve it), now the key is writing only those notes that are really meaningful.

For me, these are the 5 best ways to write notes in Google Analytics:

• Write notes with brief descriptions of your major site changes. That is, adding lots of new content, changing the website design, modifying the site navigation, and so on. After all, website analytics were designed to measure scientifically how human users and search engines navigate your site, so you have to pay special attention to strong changes in your statistics from that "major site changes" mark.
• Write a comment when any part of your homepage changes. Most returning users don't bookmark inner pages in your site: they just type the address of its homepage. And, of course, most new users are guided straight to your homepage. On top of that, your homepage is usually the main page rank distributor to the rest of your site. So any subtle change on your homepage could deeply impact both your users and search engines. And that's why you should pay attention after the "homepage changed" note in Google Analytics.
• Add a new note when any promotional campaign starts. When you put any special effort in a promotional campaign (either AdSense PPC, e-mailing, link building, and so on), you have to measure its impact. Is your campaign worth your time and effort? Does it attract new traffic? Now, you just have to look at your Analytics after the "promotional campaign started" annotation.
• Add annotations to your Analytics whenever you publish a new article, blog post or site page. If you really have the good habit of posting high quality content each day, this amount of notes could become overwhelming. Nevertheless, if you just update your contents once a week or less, it is important to measure how your content writing efforts affect your analytics. Check the "new content title" tag inside your Google Analytics graph and discover what is your best kind of content for traffic building purposes.
• Write a note whenever you detect a traffic spike in your statistics. That is, when a lot of users suddenly access your site, you register a peak in the graphs of Google Analytics. Finding the cause of such spikes is usually very easy: just look at your traffic sources. It will mostly be caused by a link pointing to your site from a very popular site. This way, these eye-catching traffic spikes won't distract you too much when understanding the subtle meaning of your Google Analytics statistics, now being marked with the "traffic spike cause" note below. And keep an eye trying to repeat such useful spikes.

Starred notes in Google Analytics

Specially meaningful annotations can be marked with a star icon, so you can filter out such less-important notes by selecting the starred option.

My suggestion is that you should try to mark everything in your statistics using these Google Analytics notes, because that will give you lots of fine-grain information. Couple this with the new Google Analytics Intelligence feature and you will be discovering tiny details that can really influence your results.

Nevertheless, Analytics notes are a time-saver feature. If you want to perform a broader analysis than the default last month statistics glance (let's say, a one year analytics report), you won't need lots of annotations - trust the most important, starred notes to get the big picture.

Google Analytics notes are a great way to understand and at glance the results of your Internet marketing efforts. Quick statistical understanding is the key to measure and optimize. So take advantage of this new time saving feature of Google Analytics when tracking your statistics. Use it wisely, following these best 5 ways to use Google Analytics notes

From the original online marketing consulting report by E. Serrano

1 comments

Permalink

Posted by ESV

Google improves long tail searches

Google boosts long tail searches with 2 improvements

Google has recently posted at its official blog 2 new search result improvements. These improvements are essentially as follows:

• An improved algorithm to suggest related searches, which will simplify how users refine the search towards more specialized terms (that is, search queries composed of more search keywords).
• Longer descriptions at the search result snippets under the title of the pages when the user inputs a long search query (with more than 3 search keywords). This makes sense, as the snippets provided with longer queries could be too short to understand the keyword context.

The increased importance of long tail searches and keywords

And, what's the main change in Google from an SEO point of view? Basically, these new improvements in Google search results will increase the effectivity of the searches composed of many search terms (the so called long tail searches), because the user will find the results easier to find and more relevant.

A search performed with more than 3 keywords is considered a very specific long tail search and thus:

• With such specific queries, you will be facing less competing pages and it will be easier to achieve good positions in search results.
• The bounce rate of such specific queries tends to be low: the search terms are so refined that the content has to be exactly what the user is expecting.
• Long tail searches are the kind of searches with the highest conversion rate at e-commerce websites: the potential customer knows perfectly what he is looking for.

So, do you have information about a very specific subject and want to create a webpage about it? Now that Google improves how users will manage search results of these kind of searches, it's the time to boost your most accurately targeted webpages.

Besides long tail searches with lots of keywords are not so frequent, you shouldn't underestimate the importance of these specific and refined queries, specially now that Google boosts these long tail searches.

0 comments

Permalink

Posted by ESV

Find users frequent questions

Find you users' frequent questions

We recently saw that only webpages with quality content will be successful in the new Search Engine Optimization scenarios. But you cannot write quality content if you don't know what your users need.

A very good way to give your users what they need is finding out their most frequent questions. What do they want to know that is related to your business? Answer such questions, provide useful content and earn your potential customer's trust.

This question search tool is a quick way to find questions related to a specific topic. You should try searching questions concerning the following subjects:

• The main keywords of your site.
• Your offered services: what do you sell / build / provide / talk about.
• Your brand name, and other similar brands.
• The name, brand or model of your products.
• Related tools or accesories.

Remember these quality content writing tips. Discover new and unsuspected user needs through frequent user questions, and satisfy these needs, building this way high quality content.

0 comments

Permalink

Posted by ESV

The end of SEO

Google SearchWiki allows you to customize search results

Have you ever typed a simple search query and then ended in a webpage whose short description seemed pretty relevant in the top of the search engine results, but which was in fact full of useless content? We have all gone through this. And now, this is coming to an end.

Wouldn't be great to have a search engine which adapted itself to your preferences? This is what Google has finally launched as Google Search Wiki: a major update of their search engine that allows users to select their favourite search results and take a look at how other users are customizing their searches.

2 innocent Google buttons that will change search forever

Starting to use Google SearchWiki is as simple as logging to an existing Google Account. The next time you perform a search in Google, you will see 2 new buttons near each specific search result:

• Promote button: moves the search result to the top of the 1st page.
• Delete button: removes the search result from the list.

A single click gives you the power to customize your search engine results, so Google will work better for you.

Do you hate these websites that seem to have the answer to your question and then lead you to a payment details page? Now you can remove such useless results forever.

On the other hand, this is the way to keep those useful but hard-to-find webpages with tons of interesting contents. Once you found something useful, you can promote it to make it easier to find again (without cluttering your browser dependent favourites list).

Will the SearchWiki Google update have some real impact?

It isn't the first time that someone tries to create customized and user community based search results. But it's the first time that the #1 search engine of the world starts doing this. So this time we are taking the largest user community and the most advanced search algorithms as an starting point.

Trust me: this searching wiki is going to be something serious. We are going to see a strong modification of the click through rate of most websites very soon with Google SearchWiki.

Nevertheless, it will take some time till this first version of the custom search results is fully established. Google will need to gather and process tons of new data from SearchWiki users. And the end users will have to get used to log into their Google accounts, reward the interesting search results, and remove the useless ones.

But, weren't many users already logged in while they checked their e-mails through Gmail, their documents through Google docs and so on? The array of free services provided by Google is so big that most users will start customizing their search results without even noticing.

Still we don't know if Google will use the data provided by millions of users who rewarded interesting search results to raise these popular websites to the top positions in Google's global search results.

SearchWiki algorithms cannot be directly applied to either unregistered users or completely new search queries. And I'm pretty sure that these two kind of searches are more than the 80% of current Google searches. That's why optimizing these general search results would still be very important for Google.

Chances are that Google will likely take into account the automated SearchWiki user's feedback to optimize and improve its search algorithms.

The main hint pointing in this direction is that Google Search Wiki displays meaningful messages such as "You are the first person to pick this result". SearchWiki even shows you global counts of "picks" and "deletions" registered for earch single search result.

All these ideas remind a lot of some kind of search results popularity system, as social bookmarking sites like Digg are using to promote the most interesting news.

On the other hand, pages deleted by many users could be penalized and sunk to the bottom positions of the overall Google search results.

So this Google update could lead to a nice way to estimate the content quality of a random webpage, much more accurate that Microsoft's proposal of using bounce rate statistics to punish webpages in Windows Live Search results.

How can you measure the effects of this Google update?

You cannot measure the impact of this recent Google update by browsing search results, because you can only check your search engine results. Remember that each user will only see her own custom search result positions (SERPs) in SearchWiki as long as she is logged into her Google account.

Let's try to estimate what could happen to your current website statistics. Since we aren't sure about how Google may use gathered customization data to tweak global search results, we cannot predict whether your number of new visitors may decrease or increase. If you were capturing traffic without relevant content, your bounce rate would be high, and probably your new visitor traffic could decrease.

Now let's check those web metrics about user satisfaction and content quality. I bet that you will have more returning visitors and even higher conversion rates. Now your satisfied customers have an easier way to return to their favourite websites without being lost in the middle of a chaos of uninteresting search engine results.

Finally, take into account that many users won't return to the search engine results page to delete irrelevant results (unless they are opening many browser tabs). So the first pages that would be removed and filtered out are those ones with a description that seems unrelated to the search query from the user's point of view. You see? The importance of writing good titles and descriptions has just become even more critical.

Is Google Search Wiki the end of SEO?

You have to understand that SEO business shouldn't rely on either repeating the same keywords one hundred times or purchasing thousands of low quality inbound links. It has never worked this way, and now this is becoming an even bigger truth. If this was your idea of SEO, then you weren't a very good SEO at all, and your optimization skills won't be useful in this new search engine scenario.

It's a somehow funny fact that the one who is actually optimizing how the search engine works is not an SEO, but the end user. Maybe we should stop talking about SEO, and start talking of content optimization or even of developing effective communication solutions.

SEO is no longer a matter of cheating with technical details. Creating useful quality content that fulfills your user's needs is the key. Give your users what they need, so they will keep coming, they will trust you, and they will tell good things about you to the rest of the world.

At the end, only you can decide what is relevant for you. This change of Google algorithms through SearchWiki will benefit all users, as the final goal is to achieve relevant pages that meet each specific user's needs.

0 comments

Permalink

Posted by ESV

Google indexes hidden pages with no inbound links

Isolated, hidden webpages could be indexed by Google

Did you know that Google could index pages that have no inbound links? I mean, pages that have not been submitted to any search engine, that are not linked from any visible indexed link, that are not listed in any sitemap, and that are hosted in servers with directory listing disabled.

In the following lines I will explain you the complete scenario that led me to investigate whether such supposedly hidden pages could be indexed by Google.

And, what's more important, I will also explain how I think Google could be authorized to discover such webpages.

Lastly, I will explain how the distributed datacenters of Google may lead to confusing results, or even to valuable hints if you really take into account every information source.

Why I thought about how Google could find pages without inbound links

While I was creating a website for a customer, I developed an under construction version of the webpage. Then I uploaded this version to my own website for testing purposes.

Let's say that my customer main URL address was:

http://my-Customer-Domain-Name/

And that I uploaded the test webpage version to:

http://my-website-domain/my-Customer-Domain-Name/index.htm

So I thought that the only way to access that under development version of the webpage was typing the whole exact directory URL, right? Wrong! I was really shocked when I performed a search in Google, using my customer domain name as the main search keyword, and the second entry in Google search results was this test webpage hosted on my own website.

Yes, a page that I considered a hidden webpage, supposedly without any inbound link, was indexed by Google!

Was that webpage really hidden from search engines?

Okay, the fact is that Google found that kind of "hidden" webpage. It was an under construction page after all, so nobody was meant to take a look at it (not yet!). So I was really interested in finding out how Google managed to index this hidden and isolated page.

The first question is: was this webpage really hidden from search engines? Here is a list of all the basics that should be checked (and that I checked) to verify that a webpage is a hidden webpage.

• The indexed webpage had no inbound links (according to Google link search). It wasn't linked from any page from my website. And, at that moment, wasn't linked from any other webpage on the whole Internet.
• The testing webpage was never submitted to any search engine. I'm not sure about whether search engines would really crawl a suggested directory, instead of trying to crawl from the main domain index. But I'm completely sure that I didn't submit the URL of my website under development to Google.
• Nobody else could have submitted such URL. None of my co-workers did know about such test version. And I think hackers have much more interesting things to do.
• The webpage was not included in a sitemap. Yes, I have Google sitemaps and Yahoo URL lists on my website. But I checked the included URL's, and the mysteriously indexed URL was not on any sitemap or list.
• Directory listing was disabled. If you don't type the exact URL, and if there isn't an index page at the target directory, my web server won't return a list of the hosted webpages - just a lean mean "directory listing disabled" message.

Then I was pretty sure that such webpage was really hidden from search engines... And then I thought that I was giving some additional information to Google (and only to that search engine, the powerful Google) in a somehow indirect, unconscious way.

So here are the two Google services that could have been used by Google to index new webpages:

Pages with Google Analytics code may be discovered and Indexed by Google

I realized that I had Google Analytics code already installed at the testing webpage. I created a profile for a new website (in fact, my customer's final website), and I embedded the tracking code (Google's latest code, ga.js) at the webpage version under construction.

On the other hand, the installation of this tracking code wasn't complete: I never uploaded the webpage under construction to the customer's domain, and Google displayed the message "tracking not installed". And the stats graphic didn't display any visit at all.

So I think that it's possible that Google could use Google Analytics data to index new web pages (or even to tweak search results in order to improve relevance).

Anonymous navigation data sent by Google Chrome could be used to index new webpages in Google

The only other Google application that had knowledge of the existance of my hidden testing webpage was the brand new, lightning fast web browser Google Chrome. I used this web browser to try the under construction webpage version. And then I found out that I had active the option of Google Chrome which allowed this browser to send anonymous data about navigation bar search suggestions and autocompletion.

I thought that these anonymous data would be used by Google to provide more relevant and more accurate search results. What I wouldn't expected is that anonymous navigation data gathered by Google Chrome could be used to index new web pages in Google. Yes, after discarding any other option, I thought at first that Google find out my hidden webpage because I provided the URL directly on the navigation bar of Google Chrome several times.

But the hidden webpage had an inbound link!

Just when I was sure that Google had indexed a hidden webpage, I received a very interesting message from Google Webmaster Tools after renaming such "hidden webpage".

The message was an URL crawling error about a missing URL. Checking the details of this error, I found that the "hidden webpage" had been linked from my customer's website (but was not linked from there anymore).

That reminded me of some of the basic principles in web programming:

• Customers will use your website in really unexpected ways (even when the website is not already finished).
• Google has many datacenters with different information, so you cannot rely on just a search result. While a normal link query may report that there aren't any inbound links, you can obtain different results by checking the stats of your website through Google Webmaster Tools
• Google Webmaster Tools now reports broken links even when the only link to your webpage comes from an external website. After all, if the linked to you, you should keep some content in that URL.

Conclusions

It seems that you cannot rely on keeping secret an isolated web page without any inbound link if you plan to use some Google applications with it. Either Google Chrome or Google Analytics could be using gathered navigation data to index new webpages and add them to the main Google search engine: you could be authorizing them to do so.

That's not a security or privacy issue. In fact, if you really don't want search engines to crawl part of your website, tell them about that in your robots.txt file. Just keep in mind that robots.txt files are public, and that human users may easily find out complete lists of your 'secret hidden urls' by reading these files. But that's a different story.

After all, Google Chrome and Google Analytics could just be aiding Google to do its work: discovering and indexing as many webpages as possible (even when those webpages seem to be really hidden!).

Finally, if you want to get accurate results about the links to your website, don't rely just on a search query. Google has many datacenters with subtle different content. So if you want serious results, perform link queries through the main Google search engine, but check the link stats of your website through Google Webmaster Tools as well.

0 comments

Permalink

Posted by ESV