20
\$\begingroup\$

WARNING: Answers may be useful to some code golfers.

In many challenges, the post contains images, which must be saved to a file in order to be able to work on the problem. This is an especially tedious manual task. We programmers should not have to be subjected to such drudgery. Your task is to automatically download all the images contained in a Code Golf.SE question.

Rules

  • Your program may connect to any part of stackexchange.com, but may not connect to any other domains, excepting the locations of the images (i.e., don't bother with a URL shortener).
  • An integer N is given as input, on the command line or stdin.
  • The URL http://codegolf.stackexchange.com/questions/N is guaranteed to be a valid link to a Code Golf question.
  • Each image displayed in the body of question N must be saved to a file on the local computer. Either of the following locations is acceptable:
    • The current directory
    • A directory input by the user
  • Your program must not save files other than the images in the question body (e.g. user avatars, or images contained in answers).
  • Images must be saved with the same file extension as the original.

This is a — write the shortest program you can.

Validity criterion for answers

There are various possible edge cases with multiple images of the same name, text with the same name as HTML elements, etc. An answer will be invalidated only if it can be shown to fail on some revision of a question posted before January 10, 2015.

\$\endgroup\$
  • \$\begingroup\$ Should the image names be kept the same or can we do like 0.png, 1.png etc \$\endgroup\$ – stokastic Jan 12 '15 at 15:57
  • \$\begingroup\$ @stokastic You can name the part before the extension to whatever you want (as long as you don't use the same name twice, overwriting a previous file). \$\endgroup\$ – feersum Jan 12 '15 at 16:06
10
\$\begingroup\$

Mathematica, 211 210 bytes

i=Import;FileNameTake@#~Export~i@#&/@ImportString["body"/.("items"/.i["http://api.stackexchange.com/2.2/questions/"<>InputString[]<>"?site=codegolf&filter=!*Lgp.gEWHA6BNP.l","JSON"])[[1]],{"HTML","ImageLinks"}]

Ungolfed:

i = Import;
FileNameTake@#~Export~i@# & /@ 
 ImportString[
  "body" /. (
    "items" /. 
      i["http://api.stackexchange.com/2.2/questions/" <> 
        InputString[] <> "?site=codegolf&filter=!*Lgp.gEWHA6BNP.l", 
       "JSON"]
  )[[1]], 
  {"HTML", "ImageLinks"}
 ]

It's pretty straightforward. I've set up a filter for the StackExchange API, which returns only the body of a question. The code retrieves the question information with that filter and parses it as JSON. I select the correct element (the body), and use ImportString to parse the HTML and filter out all image URLs. FileNameTake@#~Export~Import@# then downloads each of the images and stores it in the current working directory with the same file name as that in the URL.

You can find out the current working directory with Directory[].

In principle, there's a much shorter version, because ImportString can actually download all the files right away, instead of just giving me the URLs. But then I lose information about the original file type (since they are converted to Image objects upon download), so I can only save them all as the same type (PNG, say).

\$\endgroup\$
8
\$\begingroup\$

Javascript - 149 161 bytes

$.get("http://codegolf.stackexchange.com/q/"+prompt(),function(e){$(".post-text:first img",e).each(function(e,t){$('<a href="'+t.src+'"download>')[0].click()})})

with whitespace

$.get('http://codegolf.stackexchange.com/q/' + prompt(), function(d) {
  $('.post-text:first img',d).each(function(i,e){
   $('<a href="' + e.src + '"download>')[0].click();
  })
})

script has to be run from stackexchange site to work. Will default to the current page if no question number is specified in the prompt

\$\endgroup\$
  • 1
    \$\begingroup\$ As @doorknob mentioned above, you can save a bit by swapping q for question. And if you don't mind getting all the images in the posts on the page, you can do $('[src*="imgur"]',d) I believe. I like that this can be run in the console - instant gratification. \$\endgroup\$ – Josiah Jan 13 '15 at 2:09
  • 1
    \$\begingroup\$ questions can be shortened to q, but it should include the codegolf.stackexchange.com part instead of relying on being at that page. @Josiah it is possible to include images from other domains in posts. \$\endgroup\$ – feersum Jan 13 '15 at 2:48
  • 1
    \$\begingroup\$ The selector #question .post-text img can be shortened to .post-text:first img or .post-text:eq(0) img. \$\endgroup\$ – c.P.u1 Jan 13 '15 at 7:44
5
\$\begingroup\$

Python 2 - 241 bytes

Pretty straightforward, can probably be golfed further. I search the site for all occurrences of img src= between the first occurrence of post-text and the /div immediately following that. Each image url is then read and saved to the working directory.

import string,sys,urllib,re;o=string.find;u=urllib.urlopen
r=u("http://codegolf.stackexchange.com/q/"+sys.argv[1]).read()
i=o(r,"post-text")
for p in re.findall(r'img src="([^"]*)',r[i:o(r,"/div",i)]):f=open(p[-9:],"wb");f.write(u(p).read())
\$\endgroup\$
  • \$\begingroup\$ Filenames are kept as is - the name is taken as the last 9 bytes ([-9:]) of the image url, which should keep its 5 character name and a .png or .jpg etc. It will chop off bytes of the filename if the extension is longer than 3 characters. \$\endgroup\$ – stokastic Jan 12 '15 at 16:08
  • \$\begingroup\$ What if the file name is shorter than 9 bytes? Wouldn't that include a slash in the file name? \$\endgroup\$ – Martin Ender Jan 12 '15 at 16:09
  • \$\begingroup\$ you can save 2 bytes by making the for loop one line. for p re.findall(...):f=open(...);f.write(...) \$\endgroup\$ – undergroundmonorail Jan 12 '15 at 16:12
  • \$\begingroup\$ @mar I don't think the file name can be less than 9 bytes, but I might be mistaken \$\endgroup\$ – undergroundmonorail Jan 12 '15 at 16:13
  • \$\begingroup\$ @MartinBüttner I think 9 bytes is a reasonable assumption, but I can change it if you think I should. For what it's worth - using only 6 or 7 bytes is probably enough and will still pretty much guarantee distinct file names. \$\endgroup\$ – stokastic Jan 12 '15 at 16:16
2
\$\begingroup\$

Mathematica, 195

x=XMLElement;c=Cases;i=Import;l=Infinity;FileNameTake@#~Export~i@#&/@(((c[#,x["img",{"src"->e_,_},___]:>e,l]&)@*(c[#,x[_,{__,"id"->"question",__},e_]:>e,l]&)@*(i[#,"XMLObject"] &))@InputString[])

This exports images in the same way that Martin did in his Mathematica solution, read his answer for more information about that. This approach is very different from his, instead of parsing the result from the API I parse the HTML page directly. Or rather, I parse the symbolic XML that Mathematica can generate from HTML.

\$\endgroup\$
1
\$\begingroup\$

Python 2 - 398 342 334 bytes

The program download the SE page, extracts the post part (the post-text div element), finds urls that end in an image extension and downloads them. The images are saved as img<n>.<ext> in the current directory.

import urllib2 as u,re,sys
z=u.urlopen;i=1
p=z('http://codegolf.stackexchange.com/q/'+sys.argv[1]).read()
s=re.search(r'ss="po(.+?)/di',p,16).group(1)
for L in re.findall('"(h.+?://.*?)"',s):
 b=L.rsplit('.',1)
 if len(b)==2 and b[1].lower() in 'jpg jpeg png gif bmp'.split():
  open('img%u.%s'%(i,b[1]),'wb').write(z(L).read());i+=1

This program will also download images that are supplied as a link, not only embedded images. By giving each image a unique filename, name clashes are also avoided.

\$\endgroup\$
  • 2
    \$\begingroup\$ You can save 8 chars by replacing questions with q (in the URL). \$\endgroup\$ – Doorknob Jan 12 '15 at 16:44
  • \$\begingroup\$ In question 43274, I see only 11 images, but 21 are downloaded. \$\endgroup\$ – feersum Jan 12 '15 at 16:51
  • \$\begingroup\$ My program downloads the 10 high resolution images as well as the 10 thumbnails. I am not sure the other entries fetch the high resolution versions. \$\endgroup\$ – Logic Knight Jan 12 '15 at 17:03
  • \$\begingroup\$ @Doorknob - thanks. I missed that. I will need much more though to catch the other guys. \$\endgroup\$ – Logic Knight Jan 12 '15 at 17:09
  • 1
    \$\begingroup\$ @CarpetPython although that's arguably more useful...the intention of the spec was to download only images which are visible. \$\endgroup\$ – feersum Jan 12 '15 at 17:18
1
\$\begingroup\$

Bash - 86 bytes

wget -r -l1 -np -Ajpg,jpeg,png,bmp,gif http://codegolf.stackexchange.com/questions/$1

Nothing wget won't fix. -np prevents wget from entering upper directories(User Imgs) -A only grabs files with the extension matching the list presented. -r is a recursive download. -l prevents wget from going too deep. $1 is the question to grab.

\$\endgroup\$
  • 1
    \$\begingroup\$ Is there something specific I need to do for this to work? I tried it on a couple questions, but no good. Output here. \$\endgroup\$ – Geobits Jan 13 '15 at 3:17
  • 1
    \$\begingroup\$ I think ou can save 8 chars by replacing questions with q in the URL. \$\endgroup\$ – Timtech Jan 14 '15 at 12:13
1
\$\begingroup\$

Node.js, 251 247 Bytes

r=require,g=r('request'),g('http://codegolf.stackexchange.com/q/'+process.argv[2],function(_,_,b){r('cheerio').load(b)('#question .post-text img').each(function(i,a){s=a.attribs.src,g(s).pipe(r('fs').createWriteStream(i+r('path').basename(s)))})})

Uses request to make HTTP GETs and cheerio to parse the HTML. Name collisions are resolved by prepending the index of the current image to the basename of the file's URL. Images are saved to same directory as the current file.

\$\endgroup\$
1
\$\begingroup\$

Lua, 200 bytes

r=require'socket.http'.request r('http://codegolf.stackexchange.com/questions/'.. ...):gsub('post.text(.-)div',function(p)p:gsub('src="(.-)"',function(i)io.open(i:sub(-9),'wb'):write((r(i)))end)end)

Accepts the number as a command-line argument.

Assumes any src= attribute will be for an img tag since these are the only tags with src attributes that stack exchange allows (right?).

Also note the .. .... I'm particularly proud of that one.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.