-2
\$\begingroup\$

The Task

Write a program to take a query from STDIN and then download the first 20 search results of that query from google and save them in a folder named Search as html files. Or if google points to a non-html file (like PDF), save whatever it is. Type of results may depend on the programmer, but the first 20 results should be matched. Ads etc. can also be counted as results.

Scoring

This is ; shortest code wins.

Improve this question
\$\endgroup\$
17
  • \$\begingroup\$ You mean, retrieve the contents of whatever google points to? How should the files be named? \$\endgroup\$
    – John Dvorak
    Commented Nov 2, 2013 at 16:28
  • 1
    \$\begingroup\$ You know that this probably violates googles TOS? \$\endgroup\$
    – Johannes Kuhn
    Commented Nov 2, 2013 at 17:40
  • 3
    \$\begingroup\$ There are a couple of problems with this as it stands, not including the pointless scaling of the scoring system. "Search results" is somewhat ambiguous given the number of types of search results that Google will now give in response to a query. The instruction to "save as HTML" files is somewhat odd: surely the idea should be to save whatever the resource is, and it might not be an HTML file. \$\endgroup\$
    – Peter Taylor
    Commented Nov 2, 2013 at 18:48
  • 2
    \$\begingroup\$ @JohannesKuhn how come this is not a code golf? The score is proportional to the code length... \$\endgroup\$
    – John Dvorak
    Commented Nov 2, 2013 at 19:03
  • 1
    \$\begingroup\$ What are we supposed to do if some results fail downloading? Die spectacularly? Shrug and move on? Fetch a replacement result? I'd most preferably die spectacularly \$\endgroup\$
    – John Dvorak
    Commented Nov 3, 2013 at 6:42

2 Answers 2

Reset to default
3
\$\begingroup\$

Ruby, 260 256 characters

require'net/http';require'nokogiri';i=0;`mkdir Search`;`cd Search`;N=Net::HTTP
Nokogiri::HTML.parse(N.get"www.google.cz","/search?"+URI.encode_www_form(q:gets,num:20)).css(".r a").each{|e|File.open("#{i+=1}",?w)<<N.get(URI e['href'][/q=(.*?)&/,1])rescue 0}

Note: when testing this multiple times, be sure to add `cd ..` between each test to reset the current working directory lest you end up with whatever/Search/Search/Search/Search...

If we are allowed to fail on first unsuccessful result, we can drop rescue 0 at the end of the second line and save 8 characters.

if we may assume input is alphanumeric and a single word, we can skip encoding and shave off some more characters. (235 chars with error recovery, 227 without)

require'net/http';require'nokogiri';i=0;`mkdir Search`;`cd Search`;N=Net::HTTP
Nokogiri::HTML.parse(N.get"www.google.cz","/search?num=20&q="+gets).css(".r a").each{|e|File.open("#{i+=1}",?w)<<N.get(URI e['href'][/q=(.*?)&/,1])}
Improve this answer
\$\endgroup\$
7
  • \$\begingroup\$ Wait, is www.google.cz not the czech google? You have to use the english google, www.google.com. \$\endgroup\$
    – Kartik
    Commented Nov 3, 2013 at 8:10
  • 2
    \$\begingroup\$ Who said I have to use the english domain for Google? Does the inline link in your question count as normative? They point to the same machines with the same behavior and search results. If .com isactually required, count my solution as 261 characters. (actually, 260. I'm going to post a one-character improvement once I get to desktop and test its correctness) \$\endgroup\$
    – John Dvorak
    Commented Nov 3, 2013 at 8:37
  • \$\begingroup\$ Are you running this on Windows? \$\endgroup\$
    – manatwork
    Commented Nov 3, 2013 at 13:33
  • \$\begingroup\$ yes. Does it fail on Unix? Access rights? Unfortunately, I can't test on Linux. \$\endgroup\$
    – John Dvorak
    Commented Nov 3, 2013 at 13:36
  • \$\begingroup\$ No, the `cd Search` is pointless: the backticks are starting a child process and the directory will be changed in that – which not affects the parent process, the ruby one. \$\endgroup\$
    – manatwork
    Commented Nov 9, 2013 at 13:08
2
\$\begingroup\$

Bash

mkdir Search;cd Search;wget -U msie -rl 1 -e robots=off -d http://www.google.com http://www.google.com/search?q=`cat`&num=20

Because wget is the right way to do that.

Improve this answer
\$\endgroup\$
4
  • 1
    \$\begingroup\$ wait, does this work if the user enters a search string containing &? Shouldn't you escape the user input? \$\endgroup\$
    – John Dvorak
    Commented Nov 3, 2013 at 6:17
  • 1
    \$\begingroup\$ Maybe I should. \$\endgroup\$
    – Johannes Kuhn
    Commented Nov 3, 2013 at 11:22
  • 1
    \$\begingroup\$ Will you... probably not - too many extra characters. \$\endgroup\$
    – user8777
    Commented Nov 4, 2013 at 1:10
  • 1
    \$\begingroup\$ @LegoStormtroopr right. \$\endgroup\$
    – Johannes Kuhn
    Commented Nov 4, 2013 at 7:31

Not the answer you're looking for? Browse other questions tagged or ask your own question.