Search Syllabus

Trial version. This site is still being built, so the list and the name may change. About

Teaching or conference talk · for some experience

IR Lecture 10: Crawlers

Venkatesh Vinayakarao 58 min Published Oct 2020

This video is about 6 years old. Search advice changes quickly, so check this against current guidance.

About this video

Web crawlers are responsible for a large share of internet traffic, according to this lecture, which explains how they work, why crawling is hard and gives a short look at Nutch.

Good to know: University-style theory rather than SEO advice

Why watch it

A university-style lecture on how web crawlers work and why crawling is hard, with a short look at the open source crawler Nutch. Theory rather than SEO advice.

On the same topics