{"id":3554,"date":"2026-07-20T22:36:11","date_gmt":"2026-07-20T22:36:11","guid":{"rendered":"https:\/\/vbaseup.com\/kien-thuc\/?p=3554"},"modified":"2026-07-20T22:39:21","modified_gmt":"2026-07-20T22:39:21","slug":"crawl-budget","status":"publish","type":"post","link":"https:\/\/vbaseup.com\/kien-thuc\/crawl-budget\/","title":{"rendered":"Crawl Budget SEO: H\u01b0\u1edbng d\u1eabn t\u1ed1i \u01b0u hi\u1ec7u qu\u1ea3 cho website"},"content":{"rendered":"<p><img decoding=\"async\" src=\"https:\/\/vbaseup.com\/kien-thuc\/wp-content\/uploads\/2026\/07\/crawl-budget-seo.webp\" alt=\"Crawl budget SEO t\u1ed1i \u01b0u Googlebot thu th\u1eadp d\u1eef li\u1ec7u website\" \/><\/p>\n<p>N\u1ebfu b\u1ea1n t\u1eebng \u0111\u0103ng m\u1ed9t b\u00e0i vi\u1ebft m\u1edbi, ch\u1edd v\u00e0i ng\u00e0y, r\u1ed3i t\u00ecm t\u00ean b\u00e0i \u0111\u00f3 tr\u00ean Google m\u00e0 kh\u00f4ng th\u1ea5y \u0111\u00e2u c\u1ea3, v\u1ea5n \u0111\u1ec1 kh\u00f4ng h\u1eb3n n\u1eb1m \u1edf n\u1ed9i dung. R\u1ea5t c\u00f3 th\u1ec3 Googlebot, con b\u1ecd thu th\u1eadp d\u1eef li\u1ec7u c\u1ee7a Google, \u0111\u01a1n gi\u1ea3n l\u00e0 ch\u01b0a k\u1ecbp gh\u00e9 qua trang c\u1ee7a b\u1ea1n. \u0110\u00e2y ch\u00ednh l\u00e0 l\u00fac kh\u00e1i ni\u1ec7m crawl budget SEO xu\u1ea5t hi\u1ec7n.<\/p>\n<p>Crawl budget l\u00e0 t\u1ed5ng h\u1ee3p c\u1ee7a gi\u1edbi h\u1ea1n k\u1ef9 thu\u1eadt m\u00e0 m\u00e1y ch\u1ee7 c\u1ee7a b\u1ea1n cho ph\u00e9p v\u00e0 nhu c\u1ea7u th\u1ef1c s\u1ef1 m\u00e0 Google d\u00e0nh cho n\u1ed9i dung c\u1ee7a b\u1ea1n. Hai y\u1ebfu t\u1ed1 n\u00e0y c\u1ed9ng l\u1ea1i quy\u1ebft \u0111\u1ecbnh s\u1ed1 URL Googlebot thu th\u1eadp trong m\u1ed9t kho\u1ea3ng th\u1eddi gian nh\u1ea5t \u0111\u1ecbnh, t\u1eeb \u0111\u00f3 \u1ea3nh h\u01b0\u1edfng tr\u1ef1c ti\u1ebfp \u0111\u1ebfn vi\u1ec7c trang c\u1ee7a b\u1ea1n c\u00f3 \u0111\u01b0\u1ee3c l\u1eadp ch\u1ec9 m\u1ee5c v\u00e0 x\u1ebfp h\u1ea1ng hay kh\u00f4ng. B\u00e0i vi\u1ebft d\u01b0\u1edbi \u0111\u00e2y s\u1ebd \u0111i t\u1eeb \u0111\u1ecbnh ngh\u0129a c\u01a1 b\u1ea3n, c\u00e1ch \u0111o l\u01b0\u1eddng, \u0111\u1ebfn c\u00e1c chi\u1ebfn l\u01b0\u1ee3c t\u1ed1i \u01b0u crawl budget SEO theo t\u1eebng nh\u00f3m v\u1ea5n \u0111\u1ec1 k\u1ef9 thu\u1eadt c\u1ee5 th\u1ec3.<\/p>\n<h2>Crawl Budget l\u00e0 g\u00ec?<\/h2>\n<p><img decoding=\"async\" src=\"https:\/\/vbaseup.com\/kien-thuc\/wp-content\/uploads\/2026\/07\/crawl-budget-dinh-nghia.webp\" alt=\"Crawl budget \u0111\u1ecbnh ngh\u0129a Googlebot crawl rate limit crawl demand\" \/><\/p>\n<p>Theo Google Developers, crawl budget \u0111\u01b0\u1ee3c \u0111\u1ecbnh ngh\u0129a l\u00e0 t\u00edch h\u1ee3p c\u1ee7a hai th\u00e0nh ph\u1ea7n: crawl rate limit v\u00e0 crawl demand, t\u1ee9c l\u00e0 s\u1ed1 URL Googlebot v\u1eeba c\u00f3 kh\u1ea3 n\u0103ng v\u1eeba mu\u1ed1n thu th\u1eadp tr\u00ean m\u1ed9t website trong m\u1ed9t kho\u1ea3ng th\u1eddi gian x\u00e1c \u0111\u1ecbnh.<\/p>\n<blockquote><p>Crawl budget = crawl rate limit + crawl demand<\/p><\/blockquote>\n<p>N\u00f3i c\u00e1ch kh\u00e1c, \u0111\u00e2y kh\u00f4ng ph\u1ea3i m\u1ed9t con s\u1ed1 c\u1ed1 \u0111\u1ecbnh m\u00e0 Google c\u1ea5p cho b\u1ea1n m\u1ed7i th\u00e1ng, gi\u1ed1ng nh\u01b0 m\u1ed9t g\u00f3i data \u0111i\u1ec7n tho\u1ea1i. N\u00f3 l\u00e0 k\u1ebft qu\u1ea3 c\u1ee7a hai c\u00e2u h\u1ecfi ri\u00eang bi\u1ec7t: m\u00e1y ch\u1ee7 c\u1ee7a b\u1ea1n ch\u1ecbu t\u1ea3i \u0111\u01b0\u1ee3c bao nhi\u00eau, v\u00e0 Google c\u00f3 th\u1ea5y \u0111\u1ee7 l\u00fd do \u0111\u1ec3 gh\u00e9 th\u0103m site c\u1ee7a b\u1ea1n th\u01b0\u1eddng xuy\u00ean hay kh\u00f4ng.<\/p>\n<p>Nhi\u1ec1u ng\u01b0\u1eddi m\u1edbi b\u1eaft \u0111\u1ea7u hay nh\u1ea7m crawl budget v\u1edbi crawl rate. Th\u1ef1c ra hai kh\u00e1i ni\u1ec7m n\u00e0y kh\u00e1c nhau r\u00f5 r\u1ec7t. Crawl rate l\u00e0 t\u1ed1c \u0111\u1ed9, t\u1ee9c l\u00e0 Googlebot g\u1eedi y\u00eau c\u1ea7u nhanh hay ch\u1eadm. Crawl budget l\u00e0 t\u1ed5ng ng\u00e2n s\u00e1ch thu th\u1eadp, bao g\u1ed3m c\u1ea3 gi\u1edbi h\u1ea1n k\u1ef9 thu\u1eadt c\u1ee7a m\u00e1y ch\u1ee7 l\u1eabn nhu c\u1ea7u Google d\u00e0nh cho n\u1ed9i dung c\u1ee7a b\u1ea1n.<\/p>\n<h2>C\u00e1c th\u00e0nh ph\u1ea7n c\u1ea5u th\u00e0nh Crawl Budget<\/h2>\n<p>Tr\u01b0\u1edbc khi \u0111i v\u00e0o chi ti\u1ebft c\u00e1ch t\u1ed1i \u01b0u, b\u1ea1n c\u1ea7n hi\u1ec3u r\u00f5 hai th\u00e0nh ph\u1ea7n k\u1ef9 thu\u1eadt t\u1ea1o n\u00ean crawl budget: crawl rate limit v\u00e0 crawl demand. Ngo\u00e0i ra c\u00f2n c\u00f3 m\u1ed9t kh\u00e1i ni\u1ec7m li\u00ean quan m\u1eadt thi\u1ebft l\u00e0 crawl efficiency, quy\u1ebft \u0111\u1ecbnh vi\u1ec7c ng\u00e2n s\u00e1ch \u0111\u00f3 \u0111\u01b0\u1ee3c s\u1eed d\u1ee5ng hi\u1ec7u qu\u1ea3 \u0111\u1ebfn \u0111\u00e2u.<\/p>\n<h3>Crawl rate limit<\/h3>\n<p>Crawl rate limit x\u00e1c \u0111\u1ecbnh t\u1ea7n su\u1ea5t Googlebot g\u1eedi y\u00eau c\u1ea7u \u0111\u1ebfn m\u00e1y ch\u1ee7 c\u1ee7a b\u1ea1n m\u00e0 kh\u00f4ng g\u00e2y qu\u00e1 t\u1ea3i. Con s\u1ed1 n\u00e0y ph\u1ee5 thu\u1ed9c ch\u1ee7 y\u1ebfu v\u00e0o s\u1ee9c ch\u1ee9a c\u1ee7a m\u00e1y ch\u1ee7 (host load) v\u00e0 th\u1eddi gian ph\u1ea3n h\u1ed3i server, th\u01b0\u1eddng g\u1ecdi l\u00e0 TTFB. Googlebot t\u1ef1 \u0111\u1ed9ng \u0111i\u1ec1u ch\u1ec9nh t\u1ed1c \u0111\u1ed9 crawl d\u1ef1a tr\u00ean t\u1ea3i hi\u1ec7n t\u1ea1i c\u1ee7a m\u00e1y ch\u1ee7. Khi m\u00e1y ch\u1ee7 tr\u1ea3 v\u1ec1 l\u1ed7i 5xx ho\u1eb7c b\u1ecb timeout li\u00ean t\u1ee5c, crawl rate limit s\u1ebd gi\u1ea3m ngay l\u1eadp t\u1ee9c.<\/p>\n<p>Khi m\u00e1y ch\u1ee7 ph\u1ea3n h\u1ed3i ch\u1eadm ho\u1eb7c tr\u1ea3 v\u1ec1 l\u1ed7i 5xx th\u01b0\u1eddng xuy\u00ean, b\u1ea1n c\u00f3 th\u1ec3 th\u1ea5y Google hi\u1ec3n th\u1ecb th\u00f4ng b\u00e1o &#8220;host load exceeded&#8221; trong c\u00f4ng c\u1ee5 URL Inspection c\u1ee7a Search Console. \u0110\u00e2y l\u00e0 t\u00edn hi\u1ec7u r\u00f5 r\u00e0ng cho th\u1ea5y m\u00e1y ch\u1ee7 c\u1ee7a b\u1ea1n \u0111ang l\u00e0 \u0111i\u1ec3m ngh\u1ebdn, v\u00e0 b\u1ea1n c\u1ea7n n\u00e2ng c\u1ea5p t\u00e0i nguy\u00ean server ngay.<\/p>\n<h3>Crawl demand<\/h3>\n<p>N\u1ebfu crawl rate limit l\u00e0 c\u00e2u chuy\u1ec7n &#8220;Google c\u00f3 th\u1ec3 crawl bao nhi\u00eau&#8221;, th\u00ec crawl demand tr\u1ea3 l\u1eddi c\u00e2u h\u1ecfi &#8220;Google c\u00f3 mu\u1ed1n crawl hay kh\u00f4ng&#8221;. Crawl demand t\u0103ng l\u00ean khi URL c\u1ee7a b\u1ea1n c\u00f3 \u0111\u1ed9 ph\u1ed5 bi\u1ebfn cao, t\u1ee9c l\u00e0 nhi\u1ec1u backlink, nhi\u1ec1u l\u01b0\u1ee3t truy c\u1eadp, nhi\u1ec1u t\u00edn hi\u1ec7u chia s\u1ebb, v\u00e0 khi n\u1ed9i dung \u0111\u01b0\u1ee3c c\u1eadp nh\u1eadt th\u01b0\u1eddng xuy\u00ean. Google x\u00e1c nh\u1eadn ba y\u1ebfu t\u1ed1 ch\u00ednh t\u00e1c \u0111\u1ed9ng crawl demand l\u00e0 m\u1ee9c \u0111\u1ed9 ph\u1ed5 bi\u1ebfn, \u0111\u1ed9 m\u1edbi c\u1ee7a n\u1ed9i dung, v\u00e0 gi\u00e1 tr\u1ecb t\u1ed5ng th\u1ec3 m\u00e0 n\u1ed9i dung mang l\u1ea1i cho ng\u01b0\u1eddi d\u00f9ng.<\/p>\n<p>Ng\u01b0\u1ee3c l\u1ea1i, m\u1ed9t website c\u00f3 n\u1ed9i dung c\u0169 k\u1ef9, \u00edt backlink v\u00e0 l\u01b0\u1ee3ng traffic th\u1ea5p s\u1ebd c\u00f3 crawl demand th\u1ea5p. Khi \u0111\u00f3, Googlebot c\u00f3 xu h\u01b0\u1edbng d\u00e0nh ng\u00e2n s\u00e1ch thu th\u1eadp cho nh\u1eefng n\u01a1i kh\u00e1c \u0111ang mang l\u1ea1i gi\u00e1 tr\u1ecb cao h\u01a1n, thay v\u00ec quay l\u1ea1i site c\u1ee7a b\u1ea1n th\u01b0\u1eddng xuy\u00ean.<\/p>\n<h3>Crawl efficiency<\/h3>\n<p>Crawl efficiency l\u00e0 th\u01b0\u1edbc \u0111o hi\u1ec7u qu\u1ea3 s\u1eed d\u1ee5ng crawl budget, t\u1ee9c l\u00e0 t\u1ef7 l\u1ec7 URL th\u1ef1c s\u1ef1 c\u00f3 gi\u00e1 tr\u1ecb so v\u1edbi t\u1ed5ng s\u1ed1 URL m\u00e0 Googlebot \u0111\u00e3 b\u1ecf c\u00f4ng thu th\u1eadp. Crawl efficiency th\u1ea5p th\u01b0\u1eddng x\u1ea3y ra khi Googlebot ti\u00eau t\u1ed1n ph\u1ea7n l\u1edbn ng\u00e2n s\u00e1ch v\u00e0o c\u00e1c URL tr\u00f9ng l\u1eb7p, URL l\u1ed7i, ho\u1eb7c c\u00e1c trang c\u00f3 tham s\u1ed1 v\u00f4 ngh\u0129a, thay v\u00ec nh\u1eefng trang mang l\u1ea1i gi\u00e1 tr\u1ecb chuy\u1ec3n \u0111\u1ed5i th\u1ef1c s\u1ef1.<\/p>\n<p>M\u1ed9t \u0111i\u1ec3m quan tr\u1ecdng \u00edt ng\u01b0\u1eddi \u0111\u1ec3 \u00fd l\u00e0 Google ph\u00e2n b\u1ed5 crawl budget theo t\u1eebng s\u1ea3n ph\u1ea9m c\u1ee5 th\u1ec3 nh\u01b0 T\u00ecm ki\u1ebfm, Discover, hay Google News, ch\u1ee9 kh\u00f4ng g\u1ed9p chung th\u00e0nh m\u1ed9t con s\u1ed1 duy nh\u1ea5t. V\u00ec v\u1eady, t\u1ed1i \u01b0u crawl efficiency th\u1ef1c ch\u1ea5t l\u00e0 h\u01b0\u1edbng \u0111\u00fang lo\u1ea1i n\u1ed9i dung v\u00e0o \u0111\u00fang s\u1ea3n ph\u1ea9m m\u00e0 b\u1ea1n mu\u1ed1n xu\u1ea5t hi\u1ec7n.<\/p>\n<h2>T\u1ea1i sao Crawl Budget quan tr\u1ecdng trong SEO?<\/h2>\n<p>Khi website c\u1ee7a b\u1ea1n kh\u00f4ng \u0111\u01b0\u1ee3c Googlebot thu th\u1eadp \u0111\u1ea7y \u0111\u1ee7, h\u1ec7 qu\u1ea3 tr\u1ef1c ti\u1ebfp l\u00e0 n\u1ed9i dung m\u1edbi kh\u00f4ng \u0111\u01b0\u1ee3c l\u1eadp ch\u1ec9 m\u1ee5c k\u1ecbp th\u1eddi, \u0111\u1ed3ng ngh\u0129a v\u1edbi vi\u1ec7c b\u1ea1n m\u1ea5t c\u01a1 h\u1ed9i x\u1ebfp h\u1ea1ng v\u00e0 m\u1ea5t l\u01b0\u1ee3ng truy c\u1eadp ti\u1ec1m n\u0103ng ngay t\u1eeb nh\u1eefng ng\u00e0y \u0111\u1ea7u b\u00e0i vi\u1ebft l\u00ean s\u00f3ng.<\/p>\n<p>Crawl budget \u0111\u1eb7c bi\u1ec7t quan tr\u1ecdng v\u1edbi nh\u1eefng website quy m\u00f4 l\u1edbn, ch\u1eb3ng h\u1ea1n:<\/p>\n<ul>\n<li>Website th\u01b0\u01a1ng m\u1ea1i \u0111i\u1ec7n t\u1eed v\u1edbi h\u00e0ng ch\u1ee5c ngh\u00ecn \u0111\u1ebfn h\u00e0ng tri\u1ec7u trang s\u1ea3n ph\u1ea9m, m\u1ed7i URL b\u1ecb l\u00e3ng ph\u00ed \u0111\u1ed3ng ngh\u0129a v\u1edbi m\u1ed9t s\u1ea3n ph\u1ea9m kh\u00f4ng \u0111\u01b0\u1ee3c index v\u00e0 kh\u00f4ng th\u1ec3 b\u00e1n \u0111\u01b0\u1ee3c qua t\u00ecm ki\u1ebfm t\u1ef1 nhi\u00ean.<\/li>\n<li>Trang tin t\u1ee9c v\u00e0 portal n\u1ed9i dung l\u1edbn, n\u01a1i t\u1ed1c \u0111\u1ed9 index quy\u1ebft \u0111\u1ecbnh kh\u1ea3 n\u0103ng c\u1ea1nh tranh v\u1edbi c\u00e1c ngu\u1ed3n tin kh\u00e1c.<\/li>\n<li>Ng\u01b0\u1ee3c l\u1ea1i, website nh\u1ecf d\u01b0\u1edbi v\u00e0i tr\u0103m trang th\u01b0\u1eddng \u00edt b\u1ecb \u1ea3nh h\u01b0\u1edfng, v\u00ec Google th\u01b0\u1eddng c\u00f3 \u0111\u1ee7 kh\u1ea3 n\u0103ng crawl to\u00e0n b\u1ed9 site trong c\u00f9ng m\u1ed9t ng\u00e0y.<\/li>\n<\/ul>\n<p>Ngo\u00e0i chuy\u1ec7n index, crawl budget b\u1ecb l\u00e3ng ph\u00ed c\u00f2n k\u00e9o theo h\u1ec7 qu\u1ea3 v\u1ec1 chi ph\u00ed v\u1eadn h\u00e0nh h\u1ea1 t\u1ea7ng. Khi Googlebot li\u00ean t\u1ee5c quay l\u1ea1i thu th\u1eadp nh\u1eefng trang l\u1ed7i 404, nh\u1eefng chu\u1ed7i chuy\u1ec3n h\u01b0\u1edbng d\u00e0i, hay nh\u1eefng trang n\u1ed9i dung tr\u00f9ng l\u1eb7p, m\u00e1y ch\u1ee7 c\u1ee7a b\u1ea1n ph\u1ea3i x\u1eed l\u00fd nh\u1eefng y\u00eau c\u1ea7u v\u00f4 ngh\u0129a n\u00e0y, l\u00e0m t\u0103ng t\u1ea3i CPU v\u00e0 RAM, k\u00e9o theo TTFB t\u0103ng l\u00ean, v\u00e0 v\u00f2ng l\u1eb7p n\u00e0y l\u1ea1i ti\u1ebfp t\u1ee5c l\u00e0m gi\u1ea3m crawl rate limit.<\/p>\n<h2>C\u00e1c y\u1ebfu t\u1ed1 \u1ea3nh h\u01b0\u1edfng \u0111\u1ebfn Crawl Budget<\/h2>\n<p>C\u00f3 nhi\u1ec1u nh\u00f3m y\u1ebfu t\u1ed1 k\u1ef9 thu\u1eadt, c\u1ea3 ti\u00eau c\u1ef1c l\u1eabn t\u00edch c\u1ef1c, c\u00f9ng t\u00e1c \u0111\u1ed9ng \u0111\u1ebfn crawl budget c\u1ee7a m\u1ed9t website. Ph\u1ea7n d\u01b0\u1edbi \u0111\u00e2y s\u1ebd \u0111i qua t\u1eebng nh\u00f3m c\u1ee5 th\u1ec3, t\u1eeb c\u1ea5u tr\u00fac site \u0111\u1ebfn n\u1ed9i dung, \u0111\u1ec3 b\u1ea1n c\u00f3 th\u1ec3 t\u1ef1 \u0111\u1ed1i chi\u1ebfu v\u1edbi website c\u1ee7a m\u00ecnh.<\/p>\n<h3>K\u00edch th\u01b0\u1edbc v\u00e0 c\u1ea5u tr\u00fac website<\/h3>\n<p>S\u1ed1 l\u01b0\u1ee3ng URL tr\u00ean website c\u00e0ng l\u1edbn, Googlebot c\u00e0ng c\u1ea7n nhi\u1ec1u ng\u00e2n s\u00e1ch \u0111\u1ec3 thu th\u1eadp h\u1ebft. N\u1ebfu ng\u00e2n s\u00e1ch kh\u00f4ng \u0111\u1ee7, nh\u1eefng trang n\u1eb1m \u1edf t\u1ea7ng s\u00e2u trong c\u1ea5u tr\u00fac site, t\u1ee9c l\u00e0 ph\u1ea3i qua nhi\u1ec1u b\u01b0\u1edbc click m\u1edbi \u0111\u1ebfn \u0111\u01b0\u1ee3c, s\u1ebd c\u00f3 nguy c\u01a1 b\u1ecb Googlebot b\u1ecf qua. M\u1ed9t c\u1ea5u tr\u00fac site ph\u1eb3ng, v\u1edbi \u00edt c\u1ea5p \u0111\u1ed9 ph\u00e2n c\u1ea5p, gi\u00fap Googlebot ti\u1ebfp c\u1eadn c\u00e1c URL quan tr\u1ecdng nhanh h\u01a1n v\u00e0 ti\u1ebft ki\u1ec7m ng\u00e2n s\u00e1ch h\u01a1n.<\/p>\n<p>M\u1ed9t hi\u1ec7n t\u01b0\u1ee3ng ph\u1ed5 bi\u1ebfn g\u1ecdi l\u00e0 URL sprawl, t\u1ee9c l\u00e0 s\u1ed1 l\u01b0\u1ee3ng URL t\u0103ng kh\u00f4ng ki\u1ec3m so\u00e1t do c\u00e1c tham s\u1ed1, b\u1ed9 l\u1ecdc, hay session ID, l\u00e0 nguy\u00ean nh\u00e2n h\u00e0ng \u0111\u1ea7u khi\u1ebfn c\u00e1c website l\u1edbn m\u1ea5t ki\u1ec3m so\u00e1t crawl budget. Tr\u00ean c\u00e1c trang th\u01b0\u01a1ng m\u1ea1i \u0111i\u1ec7n t\u1eed c\u00f3 t\u00ednh n\u0103ng l\u1ecdc s\u1ea3n ph\u1ea9m theo nhi\u1ec1u ti\u00eau ch\u00ed (g\u1ecdi l\u00e0 <a href=\"https:\/\/vbaseup.com\/kien-thuc\/faceted-navigation\/\">faceted navigation<\/a>), ch\u1ec9 t\u1eeb v\u00e0i ngh\u00ecn s\u1ea3n ph\u1ea9m g\u1ed1c c\u00f3 th\u1ec3 sinh ra h\u00e0ng tri\u1ec7u bi\u1ebfn th\u1ec3 URL kh\u00e1c nhau.<\/p>\n<h3>T\u1ea7n su\u1ea5t c\u1eadp nh\u1eadt n\u1ed9i dung<\/h3>\n<p>Website \u0111\u01b0\u1ee3c c\u1eadp nh\u1eadt n\u1ed9i dung \u0111\u1ecbnh k\u1ef3, bao g\u1ed3m b\u00e0i vi\u1ebft m\u1edbi ho\u1eb7c ch\u1ec9nh s\u1eeda trang s\u1ea3n ph\u1ea9m hi\u1ec7n c\u00f3 k\u00e8m theo c\u1eadp nh\u1eadt thu\u1ed9c t\u00ednh lastmod trong sitemap, th\u01b0\u1eddng nh\u1eadn \u0111\u01b0\u1ee3c crawl demand cao h\u01a1n t\u1eeb Googlebot. \u0110\u1ed9 m\u1edbi (freshness) l\u00e0 m\u1ed9t trong ba y\u1ebfu t\u1ed1 c\u1ed1t l\u00f5i quy\u1ebft \u0111\u1ecbnh crawl demand m\u00e0 Google Developers \u0111\u00e3 x\u00e1c nh\u1eadn.<\/p>\n<p>Ng\u01b0\u1ee3c l\u1ea1i, n\u1ebfu website b\u1ecf b\u00ea vi\u1ec7c c\u1eadp nh\u1eadt n\u1ed9i dung trong th\u1eddi gian d\u00e0i, Googlebot s\u1ebd gi\u1ea3m t\u1ea7n su\u1ea5t gh\u00e9 th\u0103m. Ngay c\u1ea3 nh\u1eefng trang ch\u1ea5t l\u01b0\u1ee3ng cao c\u0169ng c\u00f3 th\u1ec3 d\u1ea7n m\u1ea5t \u0111i l\u01b0\u1ee3t crawl \u0111\u1ecbnh k\u1ef3 n\u1ebfu kh\u00f4ng c\u00f3 t\u00edn hi\u1ec7u cho th\u1ea5y n\u1ed9i dung v\u1eabn \u0111ang \u0111\u01b0\u1ee3c ch\u0103m s\u00f3c.<\/p>\n<h3>L\u1ed7i trang v\u00e0 l\u1ed7i m\u00e1y ch\u1ee7<\/h3>\n<p>C\u00e1c l\u1ed7i k\u1ef9 thu\u1eadt nh\u01b0 404, soft 404, 5xx, hay chu\u1ed7i chuy\u1ec3n h\u01b0\u1edbng \u0111\u1ec1u ti\u00eau t\u1ed1n crawl budget m\u00e0 kh\u00f4ng mang l\u1ea1i gi\u00e1 tr\u1ecb index n\u00e0o. Soft 404 \u0111\u1eb7c bi\u1ec7t nguy hi\u1ec3m h\u01a1n l\u1ed7i 404 th\u00f4ng th\u01b0\u1eddng, v\u00ec m\u00e1y ch\u1ee7 v\u1eabn tr\u1ea3 v\u1ec1 m\u00e3 HTTP 200 trong khi n\u1ed9i dung th\u1ef1c t\u1ebf l\u1ea1i tr\u1ed1ng r\u1ed7ng. Googlebot v\u1eabn crawl trang n\u00e0y nh\u01b0 b\u00ecnh th\u01b0\u1eddng, ti\u00eau t\u1ed1n ng\u00e2n s\u00e1ch, nh\u01b0ng cu\u1ed1i c\u00f9ng kh\u00f4ng c\u00f3 n\u1ed9i dung gi\u00e1 tr\u1ecb n\u00e0o \u0111\u1ec3 l\u1eadp ch\u1ec9 m\u1ee5c.<\/p>\n<p>B\u1ea3ng d\u01b0\u1edbi \u0111\u00e2y t\u00f3m t\u1eaft m\u1ee9c \u0111\u1ed9 \u1ea3nh h\u01b0\u1edfng c\u1ee7a t\u1eebng lo\u1ea1i l\u1ed7i v\u00e0 h\u01b0\u1edbng kh\u1eafc ph\u1ee5c t\u01b0\u01a1ng \u1ee9ng:<\/p>\n<table>\n<thead>\n<tr>\n<th>Lo\u1ea1i l\u1ed7i<\/th>\n<th>M\u1ee9c \u0111\u1ed9 l\u00e3ng ph\u00ed budget<\/th>\n<th>H\u00e0nh \u0111\u1ed9ng kh\u1eafc ph\u1ee5c<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>404 c\u1ee9ng<\/td>\n<td>Trung b\u00ecnh, d\u1ec5 nh\u1eadn di\u1ec7n<\/td>\n<td>X\u00f3a internal link tr\u1ecf \u0111\u1ebfn trang \u0111\u00e3 m\u1ea5t, ho\u1eb7c redirect 301 n\u1ebfu c\u00f3 trang thay th\u1ebf ph\u00f9 h\u1ee3p<\/td>\n<\/tr>\n<tr>\n<td>Soft 404<\/td>\n<td>Cao, kh\u00f3 ph\u00e1t hi\u1ec7n v\u00ec tr\u1ea3 v\u1ec1 m\u00e3 200<\/td>\n<td>Ki\u1ec3m tra n\u1ed9i dung trang tr\u1ed1ng, tr\u1ea3 v\u1ec1 \u0111\u00fang m\u00e3 404 ho\u1eb7c b\u1ed5 sung n\u1ed9i dung th\u1ef1c ch\u1ea5t<\/td>\n<\/tr>\n<tr>\n<td>5xx<\/td>\n<td>R\u1ea5t cao, \u1ea3nh h\u01b0\u1edfng c\u1ea3 crawl rate limit<\/td>\n<td>Ki\u1ec3m tra t\u00e0i nguy\u00ean server, t\u1ed1i \u01b0u hi\u1ec7u n\u0103ng backend, x\u1eed l\u00fd l\u1ed7i \u1ee9ng d\u1ee5ng<\/td>\n<\/tr>\n<tr>\n<td>Redirect chain<\/td>\n<td>Cao, m\u1ed7i b\u01b0\u1edbc chuy\u1ec3n h\u01b0\u1edbng t\u1ed1n m\u1ed9t l\u01b0\u1ee3t crawl<\/td>\n<td>R\u00fat g\u1ecdn v\u1ec1 m\u1ed9t b\u01b0\u1edbc redirect duy nh\u1ea5t t\u1eeb ngu\u1ed3n \u0111\u1ebfn \u0111\u00edch<\/td>\n<\/tr>\n<tr>\n<td>Redirect loop<\/td>\n<td>R\u1ea5t cao, Googlebot c\u00f3 th\u1ec3 ng\u1eebng crawl URL \u0111\u00f3<\/td>\n<td>Ki\u1ec3m tra v\u00e0 s\u1eeda c\u1ea5u h\u00ecnh redirect \u0111\u1ec3 tr\u00e1nh v\u00f2ng l\u1eb7p<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h3>N\u1ed9i dung tr\u00f9ng l\u1eb7p<\/h3>\n<p>N\u1ed9i dung tr\u00f9ng l\u1eb7p khi\u1ebfn Googlebot ph\u1ea3i crawl nhi\u1ec1u URL kh\u00e1c nhau cho c\u00f9ng m\u1ed9t n\u1ed9i dung th\u1ef1c ch\u1ea5t, l\u00e0m nh\u00e2n \u0111\u00f4i chi ph\u00ed crawl budget m\u00e0 kh\u00f4ng h\u1ec1 nh\u00e2n \u0111\u00f4i gi\u00e1 tr\u1ecb index. T\u00ecnh tr\u1ea1ng n\u00e0y xu\u1ea5t hi\u1ec7n t\u1eeb nhi\u1ec1u ngu\u1ed3n: phi\u00ean b\u1ea3n c\u00f3 v\u00e0 kh\u00f4ng c\u00f3 www, giao th\u1ee9c HTTP v\u00e0 HTTPS, d\u1ea5u g\u1ea1ch ch\u00e9o cu\u1ed1i URL (trailing slash), tham s\u1ed1 UTM cho m\u1ee5c \u0111\u00edch theo d\u00f5i, hay c\u00e1c trang phi\u00ean b\u1ea3n in \u1ea5n. \u0110\u00e2y c\u0169ng l\u00e0 d\u1ea5u hi\u1ec7u ph\u1ed5 bi\u1ebfn nh\u1ea5t c\u1ee7a t\u00ecnh tr\u1ea1ng <a href=\"https:\/\/vbaseup.com\/kien-thuc\/index-bloat\/\">index bloat<\/a>, t\u1ee9c l\u00e0 ch\u1ec9 m\u1ee5c c\u1ee7a website b\u1ecb ph\u00ecnh to b\u1edfi c\u00e1c URL kh\u00f4ng th\u1ef1c s\u1ef1 c\u1ea7n thi\u1ebft.<\/p>\n<p>Khi th\u1ebb canonical b\u1ecb thi\u1ebft l\u1eadp sai ho\u1eb7c b\u1ecb b\u1ecf s\u00f3t ho\u00e0n to\u00e0n, Googlebot bu\u1ed9c ph\u1ea3i t\u1ef1 quy\u1ebft \u0111\u1ecbnh \u0111\u00e2u l\u00e0 phi\u00ean b\u1ea3n g\u1ed1c. Qu\u00e1 tr\u00ecnh t\u1ef1 ph\u00e1n \u0111o\u00e1n n\u00e0y ti\u00eau t\u1ed1n th\u00eam t\u00e0i nguy\u00ean crawl, v\u00e0 trong nhi\u1ec1u tr\u01b0\u1eddng h\u1ee3p c\u00f2n d\u1eabn \u0111\u1ebfn vi\u1ec7c Google ch\u1ecdn nh\u1ea7m phi\u00ean b\u1ea3n \u0111\u1ec3 index thay v\u00ec trang b\u1ea1n th\u1ef1c s\u1ef1 mu\u1ed1n x\u1ebfp h\u1ea1ng.<\/p>\n<h3>Sitemap v\u00e0 robots.txt<\/h3>\n<p>Sitemap v\u00e0 robots.txt l\u00e0 hai c\u00f4ng c\u1ee5 ch\u1ee7 \u0111\u1ed9ng gi\u00fap b\u1ea1n \u0111\u1ecbnh h\u01b0\u1edbng crawl budget, cho Google bi\u1ebft \u0111\u00e2u l\u00e0 n\u01a1i c\u1ea7n \u01b0u ti\u00ean thu th\u1eadp v\u00e0 \u0111\u00e2u l\u00e0 n\u01a1i n\u00ean b\u1ecf qua. N\u1ebfu sitemap ch\u1ee9a nh\u1eefng URL l\u1ed7i, URL \u0111\u00e3 g\u1eafn noindex, ho\u1eb7c URL kh\u00f4ng ph\u1ea3i b\u1ea3n canonical, Googlebot s\u1ebd b\u1ecb nh\u1ea7m l\u1eabn, l\u00e0m t\u0103ng chi ph\u00ed crawl trong khi ch\u1ea5t l\u01b0\u1ee3ng index l\u1ea1i gi\u1ea3m xu\u1ed1ng. Nguy\u00ean t\u1eafc c\u01a1 b\u1ea3n l\u00e0 sitemap ch\u1ec9 n\u00ean ch\u1ee9a c\u00e1c URL tr\u1ea3 v\u1ec1 m\u00e3 HTTP 200, l\u00e0 b\u1ea3n canonical, v\u00e0 kh\u00f4ng b\u1ecb g\u1eafn noindex.<\/p>\n<p>\u1ede chi\u1ec1u ng\u01b0\u1ee3c l\u1ea1i, n\u1ebfu robots.txt v\u00f4 t\u00ecnh ch\u1eb7n (Disallow) nh\u1eefng file CSS ho\u1eb7c JavaScript c\u1ea7n thi\u1ebft \u0111\u1ec3 render trang, Googlebot s\u1ebd kh\u00f4ng th\u1ec3 hi\u1ec3u \u0111\u00fang c\u1ea5u tr\u00fac v\u00e0 giao di\u1ec7n th\u1ef1c t\u1ebf c\u1ee7a trang, \u1ea3nh h\u01b0\u1edfng \u0111\u1ebfn c\u1ea3 kh\u1ea3 n\u0103ng crawl l\u1eabn \u0111\u1ed9 ch\u00ednh x\u00e1c khi l\u1eadp ch\u1ec9 m\u1ee5c.<\/p>\n<h2>D\u1ea5u hi\u1ec7u website \u0111ang l\u00e3ng ph\u00ed Crawl Budget<\/h2>\n<p>Nh\u1eadn di\u1ec7n s\u1edbm c\u00e1c tri\u1ec7u ch\u1ee9ng l\u00e3ng ph\u00ed crawl budget gi\u00fap b\u1ea1n \u01b0u ti\u00ean \u0111\u00fang h\u00e0nh \u0111\u1ed9ng kh\u1eafc ph\u1ee5c thay v\u00ec d\u00e0n tr\u1ea3i ngu\u1ed3n l\u1ef1c. D\u1ea5u hi\u1ec7u r\u00f5 r\u00e0ng nh\u1ea5t l\u00e0 khi trang m\u1edbi ho\u1eb7c trang v\u1eeba c\u1eadp nh\u1eadt kh\u00f4ng xu\u1ea5t hi\u1ec7n tr\u00ean Google T\u00ecm ki\u1ebfm sau nhi\u1ec1u ng\u00e0y, d\u00f9 n\u1ed9i dung kh\u00f4ng c\u00f3 v\u1ea5n \u0111\u1ec1 g\u00ec v\u1ec1 ch\u1ea5t l\u01b0\u1ee3ng.<\/p>\n<p>M\u1ed9t s\u1ed1 d\u1ea5u hi\u1ec7u c\u1ee5 th\u1ec3 kh\u00e1c b\u1ea1n n\u00ean theo d\u00f5i:<\/p>\n<ul>\n<li>S\u1ed1 URL \u0111\u01b0\u1ee3c crawl m\u1ed7i ng\u00e0y th\u1ea5p h\u01a1n h\u1eb3n t\u1ed5ng s\u1ed1 URL b\u1ea1n mu\u1ed1n \u0111\u01b0\u1ee3c index.<\/li>\n<li>T\u1ef7 l\u1ec7 l\u1ed7i crawl cao trong b\u00e1o c\u00e1o c\u1ee7a Search Console.<\/li>\n<li>Nhi\u1ec1u URL ch\u1ee9a tham s\u1ed1 xu\u1ea5t hi\u1ec7n d\u00e0y \u0111\u1eb7c trong log crawl.<\/li>\n<li>C\u00e1c trang s\u1ea3n ph\u1ea9m ho\u1eb7c trang quan tr\u1ecdng kh\u00f4ng ghi nh\u1eadn l\u01b0\u1ee3t crawl n\u00e0o trong su\u1ed1t 30 ng\u00e0y.<\/li>\n<\/ul>\n<p>Tr\u00ean th\u1ef1c t\u1ebf, ph\u00e2n t\u00edch log file t\u1ea1i nhi\u1ec1u website th\u01b0\u01a1ng m\u1ea1i \u0111i\u1ec7n t\u1eed \u0111i\u1ec3n h\u00ecnh cho th\u1ea5y Googlebot c\u00f3 th\u1ec3 d\u00e0nh \u0111\u1ebfn 40 \u0111\u1ebfn 60 ph\u1ea7n tr\u0103m t\u1ed5ng l\u01b0\u1ee3t crawl v\u00e0o c\u00e1c URL ch\u1ee9a tham s\u1ed1, trang l\u1ecdc, ho\u1eb7c trang ph\u00e2n trang, thay v\u00ec c\u00e1c trang s\u1ea3n ph\u1ea9m c\u00f3 gi\u00e1 tr\u1ecb chuy\u1ec3n \u0111\u1ed5i th\u1ef1c s\u1ef1.<\/p>\n<h2>C\u00e1ch \u0111o l\u01b0\u1eddng Crawl Budget hi\u1ec7n t\u1ea1i<\/h2>\n<p>Tr\u01b0\u1edbc khi b\u1eaft tay v\u00e0o t\u1ed1i \u01b0u, b\u1ea1n c\u1ea7n bi\u1ebft ch\u00ednh x\u00e1c crawl budget hi\u1ec7n t\u1ea1i c\u1ee7a website \u0111ang \u0111\u01b0\u1ee3c s\u1eed d\u1ee5ng nh\u01b0 th\u1ebf n\u00e0o. C\u00f3 ba ph\u01b0\u01a1ng ph\u00e1p ch\u00ednh \u0111\u1ec3 l\u00e0m vi\u1ec7c n\u00e0y: b\u00e1o c\u00e1o Coverage, b\u00e1o c\u00e1o Crawl Stats, v\u00e0 ph\u00e2n t\u00edch log server.<\/p>\n<h3>Google Search Console \u2013 Coverage<\/h3>\n<p>B\u00e1o c\u00e1o Coverage trong Search Console cho b\u1ea1n bi\u1ebft URL n\u00e0o \u0111\u00e3 \u0111\u01b0\u1ee3c index, URL n\u00e0o b\u1ecb lo\u1ea1i tr\u1eeb, v\u00e0 l\u00fd do v\u00ec sao. \u0110\u00e2y l\u00e0 \u0111i\u1ec3m kh\u1edfi \u0111\u1ea7u t\u1ed1t nh\u1ea5t \u0111\u1ec3 \u0111\u00e1nh gi\u00e1 hi\u1ec7u qu\u1ea3 s\u1eed d\u1ee5ng crawl budget. B\u00e1o c\u00e1o n\u00e0y ph\u00e2n lo\u1ea1i URL theo b\u1ed1n tr\u1ea1ng th\u00e1i: Valid, Valid with warnings, Excluded, v\u00e0 Error. Trong \u0111\u00f3, nh\u1eefng URL r\u01a1i v\u00e0o tr\u1ea1ng th\u00e1i &#8220;Excluded&#8221; v\u1edbi l\u00fd do &#8220;Crawled, currently not indexed&#8221; l\u00e0 t\u00edn hi\u1ec7u cho th\u1ea5y Googlebot \u0111\u00e3 b\u1ecf c\u00f4ng crawl nh\u01b0ng kh\u00f4ng th\u1ea5y \u0111\u1ee7 gi\u00e1 tr\u1ecb \u0111\u1ec3 \u0111\u01b0a v\u00e0o ch\u1ec9 m\u1ee5c.<\/p>\n<p>M\u1ed9t c\u00e1ch ki\u1ec3m tra nhanh l\u00e0 so s\u00e1nh t\u1ed5ng s\u1ed1 URL b\u1ea1n submit trong sitemap v\u1edbi t\u1ed5ng s\u1ed1 URL \u1edf tr\u1ea1ng th\u00e1i Valid trong Coverage. Kho\u1ea3ng c\u00e1ch c\u00e0ng l\u1edbn gi\u1eefa hai con s\u1ed1 n\u00e0y, m\u1ee9c \u0111\u1ed9 l\u00e3ng ph\u00ed crawl budget c\u00e0ng nghi\u00eam tr\u1ecdng. V\u00ed d\u1ee5, n\u1ebfu sitemap c\u1ee7a b\u1ea1n ch\u1ee9a 10.000 URL nh\u01b0ng Coverage ch\u1ec9 ghi nh\u1eadn 3.000 URL h\u1ee3p l\u1ec7, \u0111i\u1ec1u \u0111\u00f3 c\u00f3 ngh\u0129a kho\u1ea3ng 70 ph\u1ea7n tr\u0103m n\u1ed7 l\u1ef1c crawl \u0111ang kh\u00f4ng mang l\u1ea1i k\u1ebft qu\u1ea3 g\u00ec.<\/p>\n<h3>Google Search Console \u2013 Crawl Stats<\/h3>\n<p>B\u00e1o c\u00e1o Crawl Stats cung c\u1ea5p d\u1eef li\u1ec7u t\u1ed5ng h\u1ee3p v\u1ec1 ho\u1ea1t \u0111\u1ed9ng crawl theo t\u1eebng ng\u00e0y, ph\u00e2n lo\u1ea1i theo m\u00e3 ph\u1ea3n h\u1ed3i HTTP, lo\u1ea1i file, v\u00e0 t\u00ecnh tr\u1ea1ng m\u00e1y ch\u1ee7 (host status). C\u1ee5 th\u1ec3, b\u00e1o c\u00e1o n\u00e0y cho b\u1ea1n th\u1ea5y:<\/p>\n<ul>\n<li>T\u1ed5ng s\u1ed1 y\u00eau c\u1ea7u crawl m\u1ed7i ng\u00e0y.<\/li>\n<li>Ph\u00e2n ph\u1ed1i theo m\u00e3 ph\u1ea3n h\u1ed3i, bao g\u1ed3m 200, 301, 404, v\u00e0 c\u00e1c l\u1ed7i 5xx.<\/li>\n<li>Lo\u1ea1i file \u0111\u01b0\u1ee3c crawl, t\u1eeb HTML, h\u00ecnh \u1ea3nh, \u0111\u1ebfn CSS v\u00e0 JavaScript.<\/li>\n<li>T\u00ecnh tr\u1ea1ng kh\u1ea3 d\u1ee5ng c\u1ee7a host trong 90 ng\u00e0y g\u1ea7n nh\u1ea5t.<\/li>\n<\/ul>\n<p>N\u1ebfu b\u1ea1n th\u1ea5y xu h\u01b0\u1edbng crawl gi\u1ea3m \u0111\u1ed9t ng\u1ed9t trong b\u00e1o c\u00e1o n\u00e0y, \u0111\u00f3 th\u01b0\u1eddng l\u00e0 d\u1ea5u hi\u1ec7u c\u1ee7a v\u1ea5n \u0111\u1ec1 m\u00e1y ch\u1ee7 ho\u1eb7c m\u1ed9t thay \u0111\u1ed5i c\u1ea5u h\u00ecnh robots.txt v\u00f4 t\u00ecnh ch\u1eb7n Googlebot ngo\u00e0i \u00fd mu\u1ed1n. Tr\u01b0\u1eddng h\u1ee3p n\u00e0y c\u1ea7n ki\u1ec3m tra ngay, \u0111\u1eb7c bi\u1ec7t n\u1ebfu s\u1ef1 s\u1ee5t gi\u1ea3m x\u1ea3y ra ngay sau m\u1ed9t l\u1ea7n deploy ho\u1eb7c c\u1eadp nh\u1eadt c\u1ea5u h\u00ecnh server.<\/p>\n<h3>Ph\u00e2n t\u00edch log server<\/h3>\n<p>Ph\u00e2n t\u00edch log file l\u00e0 ph\u01b0\u01a1ng ph\u00e1p ch\u00ednh x\u00e1c nh\u1ea5t \u0111\u1ec3 \u0111o crawl budget th\u1ef1c t\u1ebf, v\u00ec n\u00f3 cho th\u1ea5y t\u1eebng URL c\u1ee5 th\u1ec3 m\u00e0 Googlebot \u0111\u00e3 crawl, t\u1ea7n su\u1ea5t crawl, v\u00e0 m\u00e3 ph\u1ea3n h\u1ed3i t\u1ea1i \u0111\u00fang th\u1eddi \u0111i\u1ec3m \u0111\u00f3. Server log ghi l\u1ea1i to\u00e0n b\u1ed9 y\u00eau c\u1ea7u HTTP t\u1eeb Googlebot, bao g\u1ed3m URL, th\u1eddi gian, m\u00e3 ph\u1ea3n h\u1ed3i, v\u00e0 TTFB, cho ph\u00e9p b\u1ea1n x\u00e1c \u0111\u1ecbnh ch\u00ednh x\u00e1c ng\u00e2n s\u00e1ch crawl \u0111ang b\u1ecb l\u00e3ng ph\u00ed \u1edf \u0111\u00e2u. \u0110\u00e2y \u0111\u01b0\u1ee3c xem l\u00e0 ti\u00eau chu\u1ea9n trong c\u00e1c \u0111\u1ee3t <a href=\"https:\/\/vbaseup.com\/kien-thuc\/technical-seo\/\">technical SEO audit<\/a> tr\u00ean nh\u1eefng website quy m\u00f4 l\u1edbn, v\u00ec b\u00e1o c\u00e1o Crawl Stats ch\u1ec9 cho d\u1eef li\u1ec7u t\u1ed5ng h\u1ee3p, c\u00f2n log file cho ph\u00e9p soi \u0111\u1ebfn t\u1eebng URL ri\u00eang l\u1ebb.<\/p>\n<blockquote><p>B\u1ea1n c\u00f3 th\u1ec3 tham kh\u1ea3o chi ti\u1ebft h\u01a1n v\u1ec1 quy tr\u00ecnh n\u00e0y trong h\u01b0\u1edbng d\u1eabn <a href=\"https:\/\/vbaseup.com\/kien-thuc\/log-file-analysis\/\">ph\u00e2n t\u00edch log file<\/a>.<\/p><\/blockquote>\n<p>C\u00e1ch l\u00e0m ph\u1ed5 bi\u1ebfn l\u00e0 l\u1ecdc log theo user agent &#8220;Googlebot&#8221;, sau \u0111\u00f3 ph\u00e2n nh\u00f3m URL theo d\u1ea1ng (tham s\u1ed1, ph\u00e2n trang, b\u1ed9 l\u1ecdc, trang g\u1ed1c), \u0111\u1ec3 x\u00e1c \u0111\u1ecbnh t\u1ef7 l\u1ec7 ph\u1ea7n tr\u0103m ng\u00e2n s\u00e1ch crawl th\u1ef1c s\u1ef1 \u0111ang \u0111i \u0111\u1ebfn nh\u1eefng trang c\u00f3 gi\u00e1 tr\u1ecb. M\u1ed9t s\u1ed1 c\u00f4ng c\u1ee5 h\u1ed7 tr\u1ee3 quen thu\u1ed9c l\u00e0 Screaming Frog Log File Analyser, Splunk, ELK Stack, ho\u1eb7c \u0111\u01a1n gi\u1ea3n h\u01a1n l\u00e0 x\u1eed l\u00fd b\u1eb1ng Python hay Excel d\u1ef1a tr\u00ean file access log t\u1eeb Apache ho\u1eb7c Nginx.<\/p>\n<h2>Chi\u1ebfn l\u01b0\u1ee3c t\u1ed1i \u01b0u Crawl Budget<\/h2>\n<p>Sau khi \u0111\u00e3 \u0111o l\u01b0\u1eddng \u0111\u01b0\u1ee3c t\u00ecnh tr\u1ea1ng hi\u1ec7n t\u1ea1i, b\u01b0\u1edbc ti\u1ebfp theo l\u00e0 \u00e1p d\u1ee5ng chi\u1ebfn l\u01b0\u1ee3c t\u1ed1i \u01b0u theo \u0111\u00fang th\u1ee9 t\u1ef1 \u01b0u ti\u00ean: lo\u1ea1i b\u1ecf l\u00e3ng ph\u00ed tr\u01b0\u1edbc, sau \u0111\u00f3 m\u1edbi h\u01b0\u1edbng ng\u00e2n s\u00e1ch v\u00e0o nh\u1eefng trang mang l\u1ea1i gi\u00e1 tr\u1ecb cao nh\u1ea5t.<\/p>\n<h3>Lo\u1ea1i b\u1ecf ho\u1eb7c noindex trang tr\u00f9ng l\u1eb7p<\/h3>\n<p>\u0110\u00e2y th\u01b0\u1eddng l\u00e0 b\u01b0\u1edbc gi\u1ea3i ph\u00f3ng crawl budget nhanh v\u00e0 hi\u1ec7u qu\u1ea3 nh\u1ea5t. Th\u1ebb canonical tr\u1ecf v\u1ec1 \u0111\u00fang URL g\u1ed1c tr\u00ean m\u1ecdi bi\u1ebfn th\u1ec3, bao g\u1ed3m phi\u00ean b\u1ea3n www v\u00e0 non-www, HTTP v\u00e0 HTTPS, trailing slash, hay tham s\u1ed1 UTM, gi\u00fap Googlebot t\u1eadp trung ng\u00e2n s\u00e1ch v\u00e0o duy nh\u1ea5t m\u1ed9t phi\u00ean b\u1ea3n \u0111\u1ea1i di\u1ec7n. \u0110\u00e2y l\u00e0 t\u00edn hi\u1ec7u m\u1ea1nh nh\u1ea5t m\u00e0 Google s\u1eed d\u1ee5ng \u0111\u1ec3 h\u1ee3p nh\u1ea5t t\u00edn hi\u1ec7u crawl v\u00e0 index v\u1ec1 m\u1ed9t URL ch\u00ednh.<\/p>\n<p>V\u1edbi nh\u1eefng trang c\u00f3 n\u1ed9i dung m\u1ecfng, n\u1ed9i dung t\u1ef1 \u0111\u1ed9ng sinh, ho\u1eb7c n\u1ed9i dung mang t\u00ednh boilerplate kh\u00f4ng th\u00eam gi\u00e1 tr\u1ecb g\u00ec m\u1edbi, b\u1ea1n n\u00ean g\u1eafn noindex \u0111\u1ec3 lo\u1ea1i ch\u00fang kh\u1ecfi v\u00f2ng crawl. L\u01b0u \u00fd r\u1eb1ng noindex ch\u1ec9 lo\u1ea1i trang kh\u1ecfi ch\u1ec9 m\u1ee5c, Googlebot v\u1eabn c\u00f3 th\u1ec3 ti\u1ebfp t\u1ee5c crawl trang \u0111\u00f3. Mu\u1ed1n gi\u1ea3m h\u1eb3n crawl demand cho URL n\u00e0y, b\u1ea1n c\u1ea7n k\u1ebft h\u1ee3p noindex v\u1edbi vi\u1ec7c g\u1ee1 b\u1ecf to\u00e0n b\u1ed9 internal link tr\u1ecf \u0111\u1ebfn n\u00f3.<\/p>\n<h3>Ki\u1ec3m so\u00e1t URL parameters v\u00e0 b\u1ed9 l\u1ecdc<\/h3>\n<p>Tham s\u1ed1 URL v\u00e0 t\u00ednh n\u0103ng l\u1ecdc s\u1ea3n ph\u1ea9m theo nhi\u1ec1u ti\u00eau ch\u00ed l\u00e0 ngu\u1ed3n g\u1ed1c l\u1edbn nh\u1ea5t g\u00e2y ra hi\u1ec7n t\u01b0\u1ee3ng URL sprawl. C\u00e1c tham s\u1ed1 s\u1eafp x\u1ebfp, l\u1ecdc, session ID, hay tracking nh\u01b0 UTM v\u00e0 fbclid t\u1ea1o ra v\u00f4 s\u1ed1 bi\u1ebfn th\u1ec3 URL cho c\u00f9ng m\u1ed9t n\u1ed9i dung th\u1ef1c ch\u1ea5t, t\u1ea5t c\u1ea3 \u0111\u1ec1u ng\u1ed1n crawl budget m\u00e0 kh\u00f4ng h\u1ec1 t\u1ea1o ra trang n\u00e0o th\u1ef1c s\u1ef1 kh\u00e1c bi\u1ec7t v\u1ec1 gi\u00e1 tr\u1ecb. T\u1ea1i c\u00e1c website th\u01b0\u01a1ng m\u1ea1i \u0111i\u1ec7n t\u1eed c\u00f3 <a href=\"https:\/\/vbaseup.com\/kien-thuc\/faceted-navigation\/\">t\u00ednh n\u0103ng l\u1ecdc theo nhi\u1ec1u thu\u1ed9c t\u00ednh<\/a>, t\u1ef7 l\u1ec7 URL ch\u1ee9a tham s\u1ed1 c\u00f3 th\u1ec3 chi\u1ebfm \u0111\u1ebfn 80 \u0111\u1ebfn 90 ph\u1ea7n tr\u0103m t\u1ed5ng s\u1ed1 URL b\u1ecb crawl, trong khi ch\u1ec9 kho\u1ea3ng 10 \u0111\u1ebfn 20 ph\u1ea7n tr\u0103m th\u1ef1c s\u1ef1 l\u00e0 trang s\u1ea3n ph\u1ea9m.<\/p>\n<p>Gi\u1ea3i ph\u00e1p th\u1ef1c t\u1ebf bao g\u1ed3m: th\u1ed1ng nh\u1ea5t c\u00e1ch vi\u1ebft hoa vi\u1ebft th\u01b0\u1eddng trong URL, quy \u0111\u1ecbnh r\u00f5 r\u00e0ng v\u1ec1 trailing slash, d\u00f9ng th\u1ebb canonical tr\u1ecf t\u1eeb c\u00e1c URL ch\u1ee9a tham s\u1ed1 v\u1ec1 URL s\u1ea1ch, ho\u1eb7c \u0111\u01a1n gi\u1ea3n l\u00e0 Disallow to\u00e0n b\u1ed9 tham s\u1ed1 kh\u00f4ng c\u1ea7n thi\u1ebft ngay trong robots.txt.<\/p>\n<h3>S\u1eed d\u1ee5ng robots.txt \u0111\u1ec3 ch\u1eb7n c\u00e1c URL kh\u00f4ng c\u1ea7n thi\u1ebft<\/h3>\n<p>robots.txt l\u00e0 c\u00f4ng c\u1ee5 nhanh nh\u1ea5t \u0111\u1ec3 ng\u0103n Googlebot t\u1ed1n ng\u00e2n s\u00e1ch v\u00e0o nh\u1eefng khu v\u1ef1c kh\u00f4ng mang l\u1ea1i gi\u00e1 tr\u1ecb index. Nh\u1eefng khu v\u1ef1c n\u00ean \u0111\u01b0\u1ee3c Disallow bao g\u1ed3m trang k\u1ebft qu\u1ea3 t\u00ecm ki\u1ebfm n\u1ed9i b\u1ed9, gi\u1ecf h\u00e0ng, trang \u0111\u0103ng nh\u1eadp, trang qu\u1ea3n tr\u1ecb, c\u00e1c file JavaScript ph\u1ee5c v\u1ee5 tracking, v\u00e0 c\u00e1c script A\/B testing kh\u00f4ng \u1ea3nh h\u01b0\u1edfng \u0111\u1ebfn b\u1ed1 c\u1ee5c hi\u1ec3n th\u1ecb. \u0110\u00e2y l\u00e0 th\u1ef1c h\u00e0nh ph\u1ed5 bi\u1ebfn \u0111\u1ec3 b\u1ea3o v\u1ec7 crawl budget, \u0111\u1eb7c bi\u1ec7t tr\u00ean c\u00e1c website th\u01b0\u01a1ng m\u1ea1i \u0111i\u1ec7n t\u1eed.<\/p>\n<p>Tuy nhi\u00ean, tuy\u1ec7t \u0111\u1ed1i kh\u00f4ng n\u00ean Disallow c\u00e1c file CSS v\u00e0 JavaScript ph\u1ee5c v\u1ee5 vi\u1ec7c render b\u1ed1 c\u1ee5c trang. Googlebot c\u1ea7n crawl \u0111\u01b0\u1ee3c nh\u1eefng file n\u00e0y \u0111\u1ec3 hi\u1ec3u \u0111\u00fang c\u1ea5u tr\u00fac trang v\u00e0 render \u0111\u1ea7y \u0111\u1ee7 n\u1ed9i dung, tr\u01b0\u1edbc khi \u0111\u01b0a ra \u0111\u00e1nh gi\u00e1 v\u1ec1 ch\u1ea5t l\u01b0\u1ee3ng trang \u0111\u00f3. Ch\u1eb7n nh\u1ea7m c\u00e1c file n\u00e0y khi\u1ebfn Googlebot \u0111\u00e1nh gi\u00e1 sai n\u1ed9i dung v\u00e0 l\u00e0m gi\u1ea3m kh\u1ea3 n\u0103ng l\u1eadp ch\u1ec9 m\u1ee5c ch\u00ednh x\u00e1c.<\/p>\n<h3>T\u1ed1i \u01b0u sitemap.xml<\/h3>\n<p>M\u1ed9t sitemap s\u1ea1ch, ch\u00ednh x\u00e1c, v\u00e0 \u0111\u01b0\u1ee3c c\u1eadp nh\u1eadt \u0111\u1ec1u \u0111\u1eb7n s\u1ebd h\u01b0\u1edbng Googlebot \u0111\u1ebfn \u0111\u00fang nh\u1eefng URL c\u1ea7n crawl, \u0111\u1eb7c bi\u1ec7t hi\u1ec7u qu\u1ea3 v\u1edbi n\u1ed9i dung m\u1edbi xu\u1ea5t b\u1ea3n. Nguy\u00ean t\u1eafc c\u01a1 b\u1ea3n l\u00e0 sitemap ch\u1ec9 n\u00ean ch\u1ee9a URL tr\u1ea3 v\u1ec1 m\u00e3 HTTP 200, l\u00e0 b\u1ea3n canonical, v\u00e0 kh\u00f4ng b\u1ecb g\u1eafn noindex. Lo\u1ea1i b\u1ecf to\u00e0n b\u1ed9 URL l\u1ed7i, URL redirect, hay URL noindex kh\u1ecfi sitemap l\u00e0 b\u01b0\u1edbc v\u1ec7 sinh c\u01a1 b\u1ea3n \u0111\u1ea7u ti\u00ean b\u1ea1n n\u00ean l\u00e0m.<\/p>\n<p>\u0110\u1ed1i v\u1edbi c\u00e1c website l\u1edbn, vi\u1ec7c t\u1ea1o sitemap ri\u00eang cho n\u1ed9i dung m\u1edbi ho\u1eb7c v\u1eeba c\u1eadp nh\u1eadt, k\u1ebft h\u1ee3p v\u1edbi thu\u1ed9c t\u00ednh lastmod ch\u00ednh x\u00e1c, gi\u00fap Googlebot \u01b0u ti\u00ean crawl n\u1ed9i dung fresh thay v\u00ec ph\u1ea3i qu\u00e9t l\u1ea1i to\u00e0n b\u1ed9 site m\u1ed7i l\u1ea7n. B\u1ea1n c\u0169ng c\u00f3 th\u1ec3 ph\u00e2n lo\u1ea1i sitemap theo d\u1ea1ng n\u1ed9i dung, ch\u1eb3ng h\u1ea1n products.xml, blog.xml, categories.xml, \u0111\u1ec3 ch\u1ee7 \u0111\u1ed9ng \u0111i\u1ec1u ti\u1ebft m\u1ee9c \u0111\u1ed9 \u01b0u ti\u00ean crawl theo t\u1eebng nh\u00f3m URL.<\/p>\n<h3>S\u1eeda l\u1ed7i 404, 5xx v\u00e0 chu\u1ed7i chuy\u1ec3n h\u01b0\u1edbng<\/h3>\n<p>L\u1ed7i k\u1ef9 thu\u1eadt l\u00e0 nguy\u00ean nh\u00e2n tr\u1ef1c ti\u1ebfp l\u00e0m c\u1ea1n ki\u1ec7t crawl budget. Th\u1ee9 t\u1ef1 \u01b0u ti\u00ean x\u1eed l\u00fd n\u00ean l\u00e0: s\u1eeda l\u1ed7i 5xx tr\u01b0\u1edbc ti\u00ean, sau \u0111\u00f3 \u0111\u1ebfn chu\u1ed7i chuy\u1ec3n h\u01b0\u1edbng, cu\u1ed1i c\u00f9ng m\u1edbi \u0111\u1ebfn 404.<\/p>\n<p>L\u1ed7i 5xx nghi\u00eam tr\u1ecdng nh\u1ea5t v\u00ec n\u00f3 l\u00e0m gi\u1ea3m crawl rate limit ngay l\u1eadp t\u1ee9c. Khi li\u00ean t\u1ee5c nh\u1eadn v\u1ec1 l\u1ed7i m\u00e1y ch\u1ee7, Googlebot hi\u1ec3u r\u1eb1ng server \u0111ang qu\u00e1 t\u1ea3i v\u00e0 t\u1ef1 \u0111\u1ed9ng gi\u1ea3m t\u1ed1c \u0111\u1ed9 crawl \u0111\u1ec3 b\u1ea3o v\u1ec7 h\u1ec7 th\u1ed1ng c\u1ee7a b\u1ea1n, k\u00e9o theo to\u00e0n b\u1ed9 crawl budget b\u1ecb co l\u1ea1i d\u00f9 v\u1ea5n \u0111\u1ec1 g\u1ed1c c\u00f3 th\u1ec3 ch\u1ec9 n\u1eb1m \u1edf m\u1ed9t nh\u00f3m URL nh\u1ecf.<\/p>\n<p>Chu\u1ed7i chuy\u1ec3n h\u01b0\u1edbng, t\u1ee9c l\u00e0 khi m\u1ed9t URL ph\u1ea3i \u0111i qua nhi\u1ec1u b\u01b0\u1edbc redirect (t\u1eeb A sang B, r\u1ed3i sang C, r\u1ed3i m\u1edbi \u0111\u1ebfn D) tr\u01b0\u1edbc khi t\u1edbi \u0111\u00edch cu\u1ed1i c\u00f9ng, bu\u1ed9c Googlebot ph\u1ea3i theo nhi\u1ec1u b\u01b0\u1edbc, m\u1ed7i b\u01b0\u1edbc ti\u00eau t\u1ed1n m\u1ed9t l\u01b0\u1ee3t trong ng\u00e2n s\u00e1ch crawl. R\u00fat g\u1ecdn v\u1ec1 m\u1ed9t redirect tr\u1ef1c ti\u1ebfp t\u1eeb A th\u1eb3ng \u0111\u1ebfn D gi\u00fap ti\u1ebft ki\u1ec7m ngay ba l\u01b0\u1ee3t crawl cho m\u1ed7i URL. Trong c\u00e1c \u0111\u1ee3t technical SEO audit, chu\u1ed7i redirect v\u01b0\u1ee3t qu\u00e1 ba b\u01b0\u1edbc th\u01b0\u1eddng \u0111\u01b0\u1ee3c xem l\u00e0 v\u1ea5n \u0111\u1ec1 nghi\u00eam tr\u1ecdng c\u1ea7n x\u1eed l\u00fd ngay.<\/p>\n<h3>C\u1ea3i thi\u1ec7n t\u1ed1c \u0111\u1ed9 server v\u00e0 website<\/h3>\n<p>T\u1ed1c \u0111\u1ed9 ph\u1ea3n h\u1ed3i c\u1ee7a m\u00e1y ch\u1ee7, hay TTFB, l\u00e0 y\u1ebfu t\u1ed1 quy\u1ebft \u0111\u1ecbnh tr\u1ef1c ti\u1ebfp \u0111\u1ebfn crawl rate limit. M\u00e1y ch\u1ee7 ph\u1ea3n h\u1ed3i nhanh h\u01a1n cho ph\u00e9p Googlebot g\u1eedi th\u00eam y\u00eau c\u1ea7u m\u00e0 kh\u00f4ng lo g\u00e2y qu\u00e1 t\u1ea3i, t\u1eeb \u0111\u00f3 t\u1ed5ng s\u1ed1 URL \u0111\u01b0\u1ee3c crawl trong c\u00f9ng m\u1ed9t kho\u1ea3ng th\u1eddi gian s\u1ebd t\u0103ng l\u00ean t\u01b0\u01a1ng \u1ee9ng.<\/p>\n<p>Khi TTFB duy tr\u00ec \u1edf m\u1ee9c th\u1ea5p, d\u01b0\u1edbi 200 mili gi\u00e2y, Googlebot c\u00f3 th\u1ec3 crawl nhanh h\u01a1n m\u00e0 kh\u00f4ng k\u00edch ho\u1ea1t c\u01a1 ch\u1ebf b\u1ea3o v\u1ec7 host load, gi\u00fap crawl rate limit t\u0103ng theo. Google Developers x\u00e1c nh\u1eadn hai c\u00e1ch hi\u1ec7u qu\u1ea3 nh\u1ea5t \u0111\u1ec3 t\u0103ng crawl budget l\u00e0 b\u1ed5 sung t\u00e0i nguy\u00ean m\u00e1y ch\u1ee7, bao g\u1ed3m CPU, RAM, v\u00e0 b\u0103ng th\u00f4ng, k\u1ebft h\u1ee3p v\u1edbi vi\u1ec7c t\u1ed1i \u01b0u ch\u1ea5t l\u01b0\u1ee3ng n\u1ed9i dung \u0111\u1ec3 Google c\u00f3 \u0111\u1ed9ng l\u1ef1c crawl th\u01b0\u1eddng xuy\u00ean h\u01a1n.<\/p>\n<h3>C\u1ea3i thi\u1ec7n c\u1ea5u tr\u00fac internal link<\/h3>\n<p>Internal link ch\u00ednh l\u00e0 con \u0111\u01b0\u1eddng m\u00e0 Googlebot d\u00f9ng \u0111\u1ec3 kh\u00e1m ph\u00e1 v\u00e0 di chuy\u1ec3n gi\u1eefa c\u00e1c URL tr\u00ean website c\u1ee7a b\u1ea1n. M\u1ed9t <a href=\"https:\/\/vbaseup.com\/kien-thuc\/technical-seo\/\">c\u1ea5u tr\u00fac li\u00ean k\u1ebft n\u1ed9i b\u1ed9<\/a> t\u1ed1t s\u1ebd \u0111\u01b0a ng\u00e2n s\u00e1ch crawl \u0111\u1ebfn \u0111\u00fang nh\u1eefng trang mang gi\u00e1 tr\u1ecb cao nh\u1ea5t. Nh\u1eefng trang quan tr\u1ecdng c\u1ea7n nh\u1eadn \u0111\u01b0\u1ee3c nhi\u1ec1u internal link t\u1eeb c\u00e1c trang c\u00f3 \u0111\u1ed9 uy t\u00edn cao trong site, v\u00ec s\u1ed1 l\u01b0\u1ee3ng v\u00e0 ch\u1ea5t l\u01b0\u1ee3ng internal link t\u00e1c \u0111\u1ed9ng tr\u1ef1c ti\u1ebfp \u0111\u1ebfn t\u1ea7n su\u1ea5t crawl c\u1ee7a t\u1eebng URL. Ng\u01b0\u1ee3c l\u1ea1i, trang m\u1ed3 c\u00f4i, t\u1ee9c l\u00e0 trang kh\u00f4ng nh\u1eadn \u0111\u01b0\u1ee3c b\u1ea5t k\u1ef3 internal link n\u00e0o, h\u1ea7u nh\u01b0 s\u1ebd kh\u00f4ng \u0111\u01b0\u1ee3c Googlebot gh\u00e9 th\u0103m.<\/p>\n<p>B\u1ea1n c\u0169ng n\u00ean lo\u1ea1i b\u1ecf nh\u1eefng internal link tr\u1ecf \u0111\u1ebfn trang \u0111\u00e3 g\u1eafn noindex, trang l\u1ed7i 404, ho\u1eb7c trang s\u1eafp b\u1ecb x\u00f3a. M\u1ed7i li\u00ean k\u1ebft tr\u1ecf sai l\u00e0 m\u1ed9t l\u1ea7n Googlebot ph\u1ea3i di chuy\u1ec3n \u0111\u1ebfn m\u1ed9t URL kh\u00f4ng mang l\u1ea1i gi\u00e1 tr\u1ecb g\u00ec, v\u00e0 \u0111\u00f3 ch\u00ednh l\u00e0 m\u1ed9t l\u01b0\u1ee3t ng\u00e2n s\u00e1ch crawl b\u1ecb l\u00e3ng ph\u00ed.<\/p>\n<h3>\u01afu ti\u00ean crawl c\u00e1c trang chuy\u1ec3n \u0111\u1ed5i quan tr\u1ecdng<\/h3>\n<p>Ph\u00e2n b\u1ed5 crawl budget m\u1ed9t c\u00e1ch c\u00f3 ch\u1ee7 \u0111\u00edch ngh\u0129a l\u00e0 \u0111\u1ea3m b\u1ea3o Googlebot lu\u00f4n gh\u00e9 th\u0103m nh\u1eefng trang mang l\u1ea1i doanh thu v\u00e0 gi\u00e1 tr\u1ecb kinh doanh cao nh\u1ea5t tr\u01b0\u1edbc ti\u00ean. C\u00e1c trang danh m\u1ee5c ch\u00ednh, trang s\u1ea3n ph\u1ea9m b\u00e1n ch\u1ea1y, landing page chuy\u1ec3n \u0111\u1ed5i, hay b\u00e0i vi\u1ebft tr\u1ee5 c\u1ed9t n\u00ean \u0111\u01b0\u1ee3c \u01b0u ti\u00ean b\u1eb1ng c\u00e1ch t\u0103ng c\u01b0\u1eddng internal link t\u1eeb nh\u1eefng trang c\u00f3 \u0111\u1ed9 uy t\u00edn cao, \u0111\u1ed3ng th\u1eddi \u0111\u01b0a v\u00e0o nh\u00f3m sitemap \u01b0u ti\u00ean.<\/p>\n<ul>\n<li>X\u1ebfp h\u1ea1ng URL theo gi\u00e1 tr\u1ecb chuy\u1ec3n \u0111\u1ed5i th\u1ef1c t\u1ebf, sau \u0111\u00f3 t\u1eadp trung ngu\u1ed3n l\u1ef1c internal link v\u00e0 sitemap v\u00e0o nh\u00f3m URL c\u00f3 gi\u00e1 tr\u1ecb cao nh\u1ea5t.<\/li>\n<li>X\u00e2y d\u1ef1ng backlink ch\u1ea5t l\u01b0\u1ee3ng t\u1eeb b\u00ean ngo\u00e0i, k\u1ebft h\u1ee3p v\u1edbi vi\u1ec7c c\u1ea3i thi\u1ec7n t\u00edn hi\u1ec7u t\u01b0\u01a1ng t\u00e1c ng\u01b0\u1eddi d\u00f9ng nh\u01b0 traffic v\u00e0 m\u1ee9c \u0111\u1ed9 t\u01b0\u01a1ng t\u00e1c, \u0111\u1ec3 t\u0103ng crawl demand m\u1ed9t c\u00e1ch t\u1ef1 nhi\u00ean cho nh\u1eefng trang n\u00e0y.<\/li>\n<\/ul>\n<h3>X\u00f3a ho\u1eb7c h\u1ee3p nh\u1ea5t n\u1ed9i dung m\u1ecfng<\/h3>\n<p>N\u1ed9i dung m\u1ecfng v\u00e0 n\u1ed9i dung \u0111\u00e3 h\u1ebft h\u1ea1n s\u1eed d\u1ee5ng v\u1eabn ti\u00eau t\u1ed1n crawl budget m\u00e0 kh\u00f4ng \u0111\u00f3ng g\u00f3p gi\u00e1 tr\u1ecb index n\u00e0o. C\u00e1c trang s\u1ea3n ph\u1ea9m \u0111\u00e3 h\u1ebft h\u00e0ng, b\u00e0i vi\u1ebft l\u1ed7i th\u1eddi, trang danh m\u1ee5c tr\u1ed1ng, hay trang k\u1ebft qu\u1ea3 t\u00ecm ki\u1ebfm n\u1ed9i b\u1ed9 l\u00e0 nh\u1eefng v\u00ed d\u1ee5 \u0111i\u1ec3n h\u00ecnh. N\u1ebfu v\u1eabn c\u00f2n internal link tr\u1ecf \u0111\u1ebfn, Googlebot s\u1ebd ti\u1ebfp t\u1ee5c quay l\u1ea1i crawl \u0111\u1ecbnh k\u1ef3 nh\u1eefng trang n\u00e0y. Gi\u1ea3i ph\u00e1p chu\u1ea9n l\u00e0 tr\u1ea3 v\u1ec1 m\u00e3 410 n\u1ebfu n\u1ed9i dung kh\u00f4ng c\u00f2n t\u1ed3n t\u1ea1i v\u0129nh vi\u1ec5n, ho\u1eb7c redirect 301 v\u1ec1 trang danh m\u1ee5c li\u00ean quan g\u1ea7n nh\u1ea5t.<\/p>\n<p>Vi\u1ec7c h\u1ee3p nh\u1ea5t nhi\u1ec1u b\u00e0i vi\u1ebft m\u1ecfng c\u00f9ng ch\u1ee7 \u0111\u1ec1 th\u00e0nh m\u1ed9t b\u00e0i vi\u1ebft to\u00e0n di\u1ec7n, hay c\u00f2n g\u1ecdi l\u00e0 content consolidation, v\u1eeba gi\u00fap n\u00e2ng cao ch\u1ea5t l\u01b0\u1ee3ng n\u1ed9i dung, v\u1eeba gi\u1ea3m t\u1ed5ng s\u1ed1 URL m\u00e0 Googlebot c\u1ea7n crawl. K\u1ebft qu\u1ea3 l\u00e0 ng\u00e2n s\u00e1ch crawl \u0111\u01b0\u1ee3c s\u1eed d\u1ee5ng t\u1eadp trung h\u01a1n, \u0111\u1ed9 uy t\u00edn \u0111\u01b0\u1ee3c d\u1ed3n v\u00e0o m\u1ed9t URL duy nh\u1ea5t, v\u00e0 t\u1ea7n su\u1ea5t crawl c\u1ee7a URL \u0111\u00edch c\u0169ng t\u0103ng l\u00ean.<\/p>\n<h3>C\u1eadp nh\u1eadt n\u1ed9i dung th\u01b0\u1eddng xuy\u00ean v\u00e0 duy tr\u00ec RSS feed<\/h3>\n<p>Vi\u1ec7c c\u1eadp nh\u1eadt n\u1ed9i dung \u0111\u1ecbnh k\u1ef3, k\u1ebft h\u1ee3p v\u1edbi duy tr\u00ec RSS feed, gi\u00fap Googlebot ph\u00e1t hi\u1ec7n n\u1ed9i dung m\u1edbi ngay khi b\u1ea1n xu\u1ea5t b\u1ea3n, thay v\u00ec ph\u1ea3i ch\u1edd \u0111\u1ebfn chu k\u1ef3 crawl th\u00f4ng th\u01b0\u1eddng. Google s\u1eed d\u1ee5ng c\u1ea3 RSS ho\u1eb7c Atom feed l\u1eabn thu\u1ed9c t\u00ednh lastmod trong sitemap \u0111\u1ec3 ph\u00e1t hi\u1ec7n n\u1ed9i dung m\u1edbi, t\u1eeb \u0111\u00f3 t\u0103ng m\u1ee9c \u0111\u1ed9 \u01b0u ti\u00ean crawl cho nh\u1eefng URL v\u1eeba \u0111\u01b0\u1ee3c c\u1eadp nh\u1eadt.<\/p>\n<p>M\u1ed9t chi ti\u1ebft k\u1ef9 thu\u1eadt th\u01b0\u1eddng b\u1ecb b\u1ecf qua l\u00e0 header eTag trong ph\u1ea3n h\u1ed3i HTTP. eTag gi\u00fap Googlebot ki\u1ec3m tra nhanh xem n\u1ed9i dung c\u1ee7a m\u1ed9t URL c\u00f3 thay \u0111\u1ed5i hay kh\u00f4ng m\u00e0 kh\u00f4ng c\u1ea7n t\u1ea3i l\u1ea1i to\u00e0n b\u1ed9 trang. K\u1ebft h\u1ee3p v\u1edbi m\u00e3 ph\u1ea3n h\u1ed3i HTTP 304 (Not Modified), c\u01a1 ch\u1ebf n\u00e0y gi\u00fap ti\u1ebft ki\u1ec7m \u0111\u00e1ng k\u1ec3 b\u0103ng th\u00f4ng crawl, \u0111\u1eb7c bi\u1ec7t h\u1eefu \u00edch tr\u00ean nh\u1eefng website c\u00f3 quy m\u00f4 l\u1edbn.<\/p>\n<h2>C\u00e1c c\u00f4ng c\u1ee5 v\u00e0 ph\u01b0\u01a1ng ph\u00e1p h\u1ed7 tr\u1ee3 t\u1ed1i \u01b0u Crawl Budget<\/h2>\n<p>\u0110\u1ec3 tri\u1ec3n khai to\u00e0n b\u1ed9 quy tr\u00ecnh t\u1eeb \u0111o l\u01b0\u1eddng, ph\u00e2n t\u00edch, \u0111\u1ebfn t\u1ed1i \u01b0u crawl budget, b\u1ea1n c\u1ea7n m\u1ed9t b\u1ed9 c\u00f4ng c\u1ee5 ph\u00f9 h\u1ee3p cho t\u1eebng giai \u0111o\u1ea1n:<\/p>\n<ul>\n<li>\u0110o l\u01b0\u1eddng: Google Search Console (b\u00e1o c\u00e1o Coverage v\u00e0 Crawl Stats) l\u00e0 c\u00f4ng c\u1ee5 mi\u1ec5n ph\u00ed, b\u1eaft bu\u1ed9c ph\u1ea3i c\u00f3; k\u1ebft h\u1ee3p th\u00eam c\u00f4ng c\u1ee5 ph\u00e2n t\u00edch log ri\u00eang n\u1ebfu c\u1ea7n \u0111\u1ed9 chi ti\u1ebft cao h\u01a1n.<\/li>\n<li>Ph\u00e2n t\u00edch crawl chuy\u00ean s\u00e2u: Screaming Frog, Sitebulb, v\u1edbi Screaming Frog Log File Analyser l\u00e0 ti\u00eau chu\u1ea9n ph\u1ed5 bi\u1ebfn trong ng\u00e0nh cho vi\u1ec7c ph\u00e2n t\u00edch log file.<\/li>\n<li>Qu\u1ea3n l\u00fd sitemap: c\u00e1c plugin nh\u01b0 Yoast SEO ho\u1eb7c Rank Math n\u1ebfu website ch\u1ea1y tr\u00ean WordPress.<\/li>\n<li>Ki\u1ec3m tra t\u1ed1c \u0111\u1ed9 server: GTmetrix, PageSpeed Insights, gi\u00fap b\u1ea1n theo d\u00f5i TTFB v\u00e0 c\u00e1c ch\u1ec9 s\u1ed1 hi\u1ec7u n\u0103ng li\u00ean quan.<\/li>\n<\/ul>\n<p>M\u1ed9t quy tr\u00ecnh t\u1ed1i \u01b0u crawl budget SEO b\u00e0i b\u1ea3n th\u01b0\u1eddng g\u1ed3m b\u1ed1n b\u01b0\u1edbc: \u0111o l\u01b0\u1eddng baseline b\u1eb1ng Search Console v\u00e0 log file, ph\u00e2n lo\u1ea1i URL theo gi\u00e1 tr\u1ecb, \u00e1p d\u1ee5ng c\u00e1c bi\u1ec7n ph\u00e1p k\u1ef9 thu\u1eadt theo \u0111\u00fang th\u1ee9 t\u1ef1 \u01b0u ti\u00ean \u0111\u00e3 tr\u00ecnh b\u00e0y \u1edf tr\u00ean, v\u00e0 cu\u1ed1i c\u00f9ng l\u00e0 theo d\u00f5i Crawl Stats \u0111\u1ecbnh k\u1ef3 h\u00e0ng tu\u1ea7n \u0111\u1ec3 \u0111\u00e1nh gi\u00e1 hi\u1ec7u qu\u1ea3. V\u1edbi nh\u1eefng website quy m\u00f4 l\u1edbn, vi\u1ec7c l\u1eb7p l\u1ea1i quy tr\u00ecnh audit n\u00e0y m\u1ed7i qu\u00fd gi\u00fap b\u1ea1n ph\u00e1t hi\u1ec7n s\u1edbm c\u00e1c hi\u1ec7n t\u01b0\u1ee3ng URL sprawl m\u1edbi ph\u00e1t sinh, \u0111\u1ea3m b\u1ea3o ng\u00e2n s\u00e1ch crawl lu\u00f4n h\u01b0\u1edbng \u0111\u1ebfn nh\u1eefng trang c\u00f3 gi\u00e1 tr\u1ecb cao nh\u1ea5t.<\/p>\n<p>Suy cho c\u00f9ng, t\u1ed1i \u01b0u crawl budget SEO kh\u00f4ng ph\u1ea3i l\u00e0 m\u1ed9t h\u00e0nh \u0111\u1ed9ng l\u00e0m m\u1ed9t l\u1ea7n r\u1ed3i th\u00f4i. \u0110\u00e2y l\u00e0 m\u1ed9t quy tr\u00ecnh qu\u1ea3n l\u00fd li\u00ean t\u1ee5c, \u0111\u1ea3m b\u1ea3o Googlebot lu\u00f4n c\u00f3 th\u1ec3 thu th\u1eadp v\u00e0 l\u1eadp ch\u1ec9 m\u1ee5c \u0111\u00fang trang, \u0111\u00fang th\u1eddi \u0111i\u1ec3m, v\u1edbi ngu\u1ed3n l\u1ef1c m\u00e1y ch\u1ee7 t\u1ed1i thi\u1ec3u. M\u1ecdi chi\u1ebfn l\u01b0\u1ee3c n\u1ed9i dung, d\u00f9 c\u00f4ng phu \u0111\u1ebfn \u0111\u00e2u, c\u0169ng s\u1ebd tr\u1edf n\u00ean v\u00f4 ngh\u0129a n\u1ebfu Googlebot kh\u00f4ng th\u1ec3 ti\u1ebfp c\u1eadn v\u00e0 l\u1eadp ch\u1ec9 m\u1ee5c ch\u00fang m\u1ed9t c\u00e1ch \u0111\u00fang \u0111\u1eafn.<\/p>\n<h4>Crawl budget c\u00f3 \u1ea3nh h\u01b0\u1edfng \u0111\u1ebfn website nh\u1ecf kh\u00f4ng?<\/h4>\n<p>Kh\u00f4ng. Website nh\u1ecf d\u01b0\u1edbi v\u00e0i tr\u0103m trang th\u01b0\u1eddng kh\u00f4ng b\u1ecb \u1ea3nh h\u01b0\u1edfng \u0111\u00e1ng k\u1ec3, v\u00ec Googlebot c\u00f3 \u0111\u1ee7 kh\u1ea3 n\u0103ng crawl to\u00e0n b\u1ed9 site trong m\u1ed9t kho\u1ea3ng th\u1eddi gian ng\u1eafn. Crawl budget ch\u1ec9 th\u1ef1c s\u1ef1 tr\u1edf th\u00e0nh v\u1ea5n \u0111\u1ec1 nghi\u00eam tr\u1ecdng khi website v\u01b0\u1ee3t ng\u01b0\u1ee1ng h\u00e0ng ch\u1ee5c ngh\u00ecn URL tr\u1edf l\u00ean. Google c\u0169ng x\u00e1c nh\u1eadn r\u1eb1ng crawl budget kh\u00f4ng ph\u1ea3i \u0111i\u1ec1u c\u1ea7n lo ng\u1ea1i \u0111\u1ed1i v\u1edbi c\u00e1c site nh\u1ecf v\u00e0 trung b\u00ecnh c\u00f3 n\u1ed9i dung ch\u1ea5t l\u01b0\u1ee3ng t\u1ed1t.<\/p>\n<h4>C\u00f3 th\u1ec3 t\u1ef1 t\u0103ng crawl budget b\u1eb1ng c\u00e1ch y\u00eau c\u1ea7u Google crawl kh\u00f4ng?<\/h4>\n<p>C\u00f3, nh\u01b0ng ch\u1ec9 \u1edf m\u1ee9c \u0111\u1ed9 h\u1ea1n ch\u1ebf. Search Console cho ph\u00e9p b\u1ea1n y\u00eau c\u1ea7u crawl m\u1ed9t URL \u0111\u01a1n l\u1ebb th\u00f4ng qua c\u00f4ng c\u1ee5 URL Inspection. Tuy nhi\u00ean, t\u1ed5ng crawl budget c\u1ee7a to\u00e0n b\u1ed9 website v\u1eabn do Google quy\u1ebft \u0111\u1ecbnh d\u1ef1a tr\u00ean s\u1ee9c ch\u1ee9a m\u00e1y ch\u1ee7 v\u00e0 crawl demand t\u1ed5ng th\u1ec3. Kh\u00f4ng c\u00f3 c\u00e1ch n\u00e0o \u0111\u1ec3 \u00e9p t\u0103ng crawl budget to\u00e0n site tr\u1ef1c ti\u1ebfp t\u1eeb ph\u00eda qu\u1ea3n tr\u1ecb vi\u00ean. Google hi\u1ec7n kh\u00f4ng cung c\u1ea5p API hay c\u00f4ng c\u1ee5 n\u00e0o cho ph\u00e9p t\u0103ng crawl budget t\u1ed5ng th\u1ec3; c\u00e1ch duy nh\u1ea5t l\u00e0 c\u1ea3i thi\u1ec7n gi\u00e1n ti\u1ebfp th\u00f4ng qua s\u1ee9c ch\u1ee9a m\u00e1y ch\u1ee7 v\u00e0 ch\u1ea5t l\u01b0\u1ee3ng n\u1ed9i dung.<\/p>\n<h4>Blocking AI bots c\u00f3 gi\u00fap t\u0103ng crawl budget cho Googlebot kh\u00f4ng?<\/h4>\n<p>C\u00f3, trong m\u1ed9t s\u1ed1 tr\u01b0\u1eddng h\u1ee3p. N\u1ebfu c\u00e1c AI bot nh\u01b0 GPTBot, ClaudeBot, hay PerplexityBot \u0111ang ti\u00eau t\u1ed1n \u0111\u00e1ng k\u1ec3 t\u00e0i nguy\u00ean m\u00e1y ch\u1ee7 c\u1ee7a b\u1ea1n, vi\u1ec7c ch\u1eb7n ch\u00fang trong robots.txt s\u1ebd gi\u1ea3i ph\u00f3ng CPU, RAM, v\u00e0 b\u0103ng th\u00f4ng, t\u1eeb \u0111\u00f3 gi\u00e1n ti\u1ebfp t\u0103ng crawl rate limit cho Googlebot. Tuy nhi\u00ean, t\u00e1c \u0111\u1ed9ng n\u00e0y ch\u1ec9 th\u1ef1c s\u1ef1 c\u00f3 \u00fd ngh\u0129a khi l\u01b0u l\u01b0\u1ee3ng t\u1eeb AI bot chi\u1ebfm t\u1ef7 tr\u1ecdng l\u1edbn trong t\u1ed5ng t\u1ea3i m\u00e1y ch\u1ee7. Tr\u00ean th\u1ef1c t\u1ebf, m\u1ed9t s\u1ed1 website l\u1edbn ghi nh\u1eadn l\u01b0u l\u01b0\u1ee3ng AI bot chi\u1ebfm t\u1eeb 15 \u0111\u1ebfn 30 ph\u1ea7n tr\u0103m t\u1ed5ng s\u1ed1 y\u00eau c\u1ea7u crawl, v\u00e0 vi\u1ec7c ch\u1eb7n ch\u00fang trong robots.txt gi\u00fap gi\u1ea3m t\u1ea3i m\u00e1y ch\u1ee7 \u0111\u00e1ng k\u1ec3, \u0111\u1ed3ng th\u1eddi c\u1ea3i thi\u1ec7n TTFB cho Googlebot.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>N\u1ebfu b\u1ea1n t\u1eebng \u0111\u0103ng m\u1ed9t b\u00e0i vi\u1ebft m\u1edbi, ch\u1edd v\u00e0i ng\u00e0y, r\u1ed3i t\u00ecm t\u00ean b\u00e0i \u0111\u00f3 tr\u00ean Google m\u00e0 kh\u00f4ng th\u1ea5y \u0111\u00e2u c\u1ea3, v\u1ea5n \u0111\u1ec1 kh\u00f4ng h\u1eb3n n\u1eb1m \u1edf n\u1ed9i dung. R\u1ea5t c\u00f3 th\u1ec3 Googlebot, con b\u1ecd thu th\u1eadp d\u1eef li\u1ec7u c\u1ee7a Google, \u0111\u01a1n gi\u1ea3n l\u00e0 ch\u01b0a k\u1ecbp gh\u00e9 qua trang c\u1ee7a b\u1ea1n. \u0110\u00e2y ch\u00ednh [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3556,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14,24],"tags":[25],"class_list":["post-3554","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seo","category-technical-seo","tag-technical-seo"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/posts\/3554","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/comments?post=3554"}],"version-history":[{"count":2,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/posts\/3554\/revisions"}],"predecessor-version":[{"id":3558,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/posts\/3554\/revisions\/3558"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/media\/3556"}],"wp:attachment":[{"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/media?parent=3554"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/categories?post=3554"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vbaseup.com\/kien-thuc\/wp-json\/wp\/v2\/tags?post=3554"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}