Lanka Developers Community

    Lanka Developers

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Shop

    Basic SEO for Web Developers Part 2

    Web Development
    seo website
    4
    6
    1366
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • dev_lak
      dev_lak last edited by ciaompe

      1. Robots Exclusion Standard

      • Robots.txt
      Location: https://example.com/robots.txt
      
      User-agent: googlebot
      Disallow: /example.html
      Sitemap: https://example.com/sitemap.xml
      
      ##More information at http://www.robotstxt.org/robotstxt.html
      
      • X-Robots
      Location: Sent in the HTTP headers
      
      X-Robots-Tag: noindex
      
      ##More information at http://noarchive.net/xrobots/
      
      • Meta Robots
      Location: In the html <head>
      
      <meta name="ROBOT NAME" content="ARGUMENTS" />
      
      ##More information athttp://www.robotstxt.org/meta.html
      
      • Robots Best Practices

        1. Only Meta Robots and X-Robots remove URLs from search results
        2. Don't block CSS or JavaScript files with robots.txt
      • Arguments can be:

        1. Nofollow (do not follow links)
        2. Noindex (do not index)
        3. Noarchive (do not archive)
        4. NoODP (Do not show Open Directory Project description)...Or combined (noindex, nofollow)
        5. If the robots <META> tag is not defined, the default is "INDEX,FOLLOW"

      2. Important User Agents

      • For robots.txt, robots meta tags, and X-Robots-Tag

        Googlebot (can be used as default for most Google crawlers)
        Googlebot-News
        Googlebot-Image
        AdsBot-Google
        Baiduspider
        FacebookExternalHit
        Slurp
        Mediapartners-Google (Mobile Adsense) or Mediapartners
        Googlebot-Mobile
        Googlebot-Video
        Bingbot
        Yandexbot
        Applebot
        Twitterbot
        Rogerbot
        

      3. Sitemap Syntax

      • XML Sitemaps

        <?xml version="1.0" encoding="UTF-8"?>
        <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
            <url>
                <loc>https://example.com/</loc> 
                <lastmod>2015-01-01</lastmod> 
                <changefreq>monthly</changefreq> 
                <priority>0.9</priority>
            </url> 
        </urlset>
        
      • Sitemap Index File

        <?xml version="1.0" encoding="UTF-8"?>
        <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
            <sitemap>
                <loc>https://example.com/sitemap1.xml.gz</loc>
                <lastmod>2015-01-01T18:23:17+00:00</lastmod>
            </sitemap>
            <sitemap>
                <loc>https://example.com/sitemap2.xml.gz</loc>
                <lastmod>2015-01-01</lastmod>
            </sitemap>
        </sitemapindex>
        
      • Default Locations Can Be:

        https://example.com/sitemap.xml
        https://example.com/sitemap.xml.gz
        https://example.com/sitemap.gz
        
      • Other Common Sitemap Types:

        Mobile
        News
        Image
        Video
        

      4. Pagination

      • Use rel="next" andrel="prev" in the <head> sectionto indicate the relationship between paginated URLs

      • First Page - https://example.com/article

        <link rel="next" href="https://example.com/article?pg=2">
        
      • Second Page - https://example.com/article?pg=2

        <link rel="prev" href="https://example.com/article">
        <link rel="next" href="https://example.com/article?pg=3">
        
      • Final Page - https://example.com/article?pg=3

        <link rel="prev" href="https://example.com/article?pg=2">
        
        #More information at http://mz.cm/rel-next
        

      I'll post next part asap.
      Thank you all for your time in reading this!

      1 Reply Last reply Reply Quote 3
      • root
        root Linux Help last edited by

        Thnx bro. can you link this part with previous part.

        1 Reply Last reply Reply Quote 1
        • dev_lak
          dev_lak last edited by

          @root ur welcome, sure i'll link

          1 Reply Last reply Reply Quote 0
          • F
            fern Node.js last edited by

            Nice post... 😍

            1 Reply Last reply Reply Quote 1
            • rachitha
              rachitha last edited by

              Thank you...

              1 Reply Last reply Reply Quote 1
              • dev_lak
                dev_lak last edited by

                @fern & @rachitha ur welcome :smiley:

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post

                2
                Online

                3.6k
                Users

                1.3k
                Topics

                5.3k
                Posts

                • Privacy
                • Terms & Conditions
                • Donate

                © Copyrights and All right reserved Lanka Developers Community

                Powered by Axis Technologies (PVT) Ltd

                Made with in Sri Lanka

                | |