This lab activity asks you to complete some code provided so that it is able to crawl the web from a given starting URL. You will start by completing a sequential version of this code. Then you will work on a version that uses the libraries from java.util.concurrent to create a multi-threaded version.
Some existing knowledge of the Java data structures LinkedList and ArrayList is necessary.
Please download ConcurrentDataStructures.jar. Inside the src directory are two packages: one for the original sequential web crawler called lab/spider, and another for the threaded spider called lab/concurrentSpider. The first chapter linked below starts you out with the sequential spider using conventional Java data structures to hold the URLs it encounters. You will then continue on to create the threaded version, starting with this code provided.