Skip to main content

Timeline for Code Golf: Make a Web Crawler

Current License: CC BY-SA 3.0

9 events
when toggle format what by license comment
Mar 30, 2012 at 7:15 comment added elssar I wonder if this is a way to get some sample code for the Udacity web crawler competition which will be held after the end of Udacity's CS101 - Building a search engine - course.
Mar 30, 2012 at 0:00 answer added listenrightmeow timeline score: 5
Mar 29, 2012 at 1:46 comment added user unknown @CeilingSpy: What's about updating it? Or flag it for closing by a mod, and discuss it in chat or at the meta sandbox, which is meant for that, until clarification. It can be reopened later.
Mar 28, 2012 at 19:40 comment added 3aw5TZetdf Oh, I never actually thought about that. So do you think this question should be deleted?
Mar 28, 2012 at 18:37 comment added captncraig To clarify dmckee's comment, any considerate web crawler should find and honor the robots.txt document for any domain it crawls. One of my college courses used this as an assignment and consistantly got the campus network blacklsted from several major sites a few times a year. See robotstxt.org
Mar 28, 2012 at 14:55 comment added dmckee --- ex-moderator kitten "Although this is code golf, the highest voted answer/solution will win, and don't forget to have fun!" No. [code-golf] defines the winning critera. If you want a different objective winning critera you can set that, but you should be aware that many user don't like "most votes" as a criteria.
Mar 28, 2012 at 14:53 comment added dmckee --- ex-moderator kitten Participants are cautioned that you can make site owners and/or your ISP very mad by doing this badly.
Mar 28, 2012 at 11:39 history tweeted twitter.com/#!/StackCodeGolf/status/184967994080886785
Mar 28, 2012 at 6:59 history asked 3aw5TZetdf CC BY-SA 3.0