To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]

Many brands are now heavily using this mobile app to boost their marketing strategy. Instagram can be used to gain the necessary momentum needed to capture the attention of the market segment that has an interest in the product offering or services.[54] As Instagram is supported by Apple and android system, it can be easily accessed by smartphone users. Moreover, it can be accessed by the Internet as well. Thus, the marketers see it as a potential platform to expand their brands exposure to the public, especially the younger target group. On top of this, marketers do not only use social media for traditional Internet advertising, but they also encourage users to create attention for a certain brand. This generally creates an opportunity for greater brand exposure.[55] Furthermore, marketers are also using the platform to drive social shopping and inspire people to collect and share pictures of their favorite products. Many big names have already jumped on board: Starbucks, MTV, Nike, Marc Jacobs, and Red Bull are a few examples of multinationals that adopted the mobile photo app early. Fashion blogger Danielle Bernstein, who goes by @weworewhat on Instagram, collaborated with Harper's Bazaar to do a piece on how brands are using Instagram to market their products, and how bloggers make money from it. Bernstein, who currently has one and a half million followers on Instagram, and whose "outfit of the day" photos on Snapchat get tens of thousands of screenshots, explained that for a lot of her sponsored posts, she must feature the brand in a certain number of posts, and often cannot wear a competitor's product in the same picture. According to Harper's Bazaar, industry estimates say that brands are spending more than $1 billion per year on consumer-generated advertising.[56] Founder of Instagram Kevin Systrom even went to Paris Fashion week, going to couture shows and meeting with designers to learn more about how style bloggers, editors, and designers are currently dominating much of the content on his application.[57]

When Googlebot crawls a page, it should see the page the same way an average user does15. For optimal rendering and indexing, always allow Googlebot access to the JavaScript, CSS, and image files used by your website. If your site's robots.txt file disallows crawling of these assets, it directly harms how well our algorithms render and index your content. This can result in suboptimal rankings.
The kind of content you create depends on your audience's needs at different stages in the buyer's journey. You should start by creating buyer personas (use these free templates, or try makemypersona.com) to identify what your audience's goals and challenges are in relation to your business. On a basic level, your online content should aim to help them meet these goals, and overcome their challenges.
This involves tracking the volume of visits, leads, and customers to a website from the individual social channel. Google Analytics[110] is a free tool that shows the behavior and other information, such as demographics and device type used, of website visitors from social networks. This and other commercial offers can aid marketers in choosing the most effective social networks and social media marketing activities.
Off page SEO: This type of SEO focuses on all of the activity that takes place "off the page" when looking to optimize your website. "What activity not on my own website could affect my ranking?" You might ask. The answer is inbound links, also known as backlinks. The number of publishers that link to you, and the relative "authority" of those publishers, affect how highly you rank for the keywords you care about. By networking with other publishers, writing guest posts on these websites (and linking back to your website), and generating external attention, you can earn the backlinks you need to move your website up on all the right SERPs.
×