The World's Smallest Web Browser

Question

Backstory:

You enjoy your new programming job at a mega-multi-corporation. However, you aren't allowed to browse the web since your computer only has a CLI. They also run sweeps of all employees' hard drives, so you can't simply download a large CLI web browser. You decide to make a simple textual browser that is as small as possible so you can memorize it and type it into a temporary file every day.

Challenge:

Your task is to create a golfed web browser within a command-line interface. It should:

Take a single URL in via args or stdin
Split the directory and host components of the URL
Send a simple HTTP request to the host to request the said directory
Print the contents of any  paragraph  tags
And either exit or ask for another page

More Info:

A simple HTTP request looks like this:

GET {{path}} HTTP/1.1
Host: {{host}}
Connection: close
\n\n

Ending newlines emphasized.

A typical response looks like:

HTTP/1.1 200 OK\n
<some headers separated by newlines>
\n\n
<html>
....rest of page

Rules:

It only needs to work on port 80 (no SSL needed)
You may not use netcat
Whatever programming language is used, only low-level TCP APIs are allowed (except netcat)
You may not use GUI, remember, it's a CLI
You may not use HTML parsers, except builtin ones (BeautifulSoup is not a builtin)
Bonus!! If your program loops back and asks for another URL instead of exiting, -40 chars (as long as you don't use recursion)
No third-party programs. Remember, you can't install anything.
code-golf, so the shortest byte count wins

Can you provide a sample web page of some sort for testing this? It is difficult to find places that use :P — a spaghetto, Commented Oct 26, 2015 at 16:43
The restriction to low-level socket interfaces seems to prohibit the TCP-level APIs of most languages which have TCP-level APIs. — Peter Taylor, Commented Oct 26, 2015 at 16:58

Community · Accepted Answer · 2020-06-17 09:04:33Z

69

Pure Bash (no utilities), 200 bytes - 40 bonus = 160

while read u;do
u=${u#*//}
d=${u%%/*}
exec 3<>/dev/tcp/$d/80
echo "GET /${u#*/} HTTP/1.1
host:$d
Connection:close
">&3
mapfile -tu3 A
a=${A[@]}
a=${a#*<p>}
a=${a%</p>*}
echo "${a//<\/p>*<p>/"
"}"
done

I think this is up to the spec, ~~though of course watch out for parsing HTML using regex~~ I think the only thing worse than parsing HTML using regex is parsing HTML using shell pattern matching.

This now deals with ... spanning multiple lines. Each ... is on a separate line of output:

$ echo "http://example.com/" | ./smallbrowse.sh
This domain is established to be used for illustrative examples in documents. You may use this     domain in examples without prior coordination or asking for permission.
<a href="http://www.iana.org/domains/example">More information...</a>
$

edited Jun 17, 2020 at 9:04

CommunityBot

1

answered Oct 26, 2015 at 17:25

Digital Trauma

72.5k9 gold badges111 silver badges262 bronze badges

36

\$\begingroup\$ You need to have this memorized by tomorrow. \$\endgroup\$
– Conor O'Brien
Commented Oct 26, 2015 at 17:45
14

\$\begingroup\$ +∞ for "parsing HTML using shell pattern matching" \$\endgroup\$
– SztupY
Commented Oct 26, 2015 at 18:20
79

\$\begingroup\$ -1 because your avatar is subliminal messaging \$\endgroup\$
– TheDoctor
Commented Oct 26, 2015 at 19:48
1

\$\begingroup\$ ...you can make TCP connections from Bash? Now I am truly terrified! \$\endgroup\$
– MathematicalOrchid
Commented Oct 27, 2015 at 13:08
4

\$\begingroup\$ Note: /dev/tcp is an optional extension and may not be present in your build of bash. You need to compile with --enable-net-redirections to have it. \$\endgroup\$
– Chris Down
Commented Oct 27, 2015 at 14:45

| Show 7 more comments

Community · Accepted Answer · 2020-06-17 09:04:33Z

22

PHP, 175 bytes (215 - 40 bonus) 227 229 239 202 216 186 bytes

Have fun browsing the web:

for(;$i=parse_url(trim(fgets(STDIN))),fwrite($f=fsockopen($h=$i[host],80),"GET $i[path] HTTP/1.1
Host:$h
Connection:Close

");preg_match_all('!<p>(.+?)</p>!si',stream_get_contents($f),$r),print join("
",$r[1])."
");

Reads URLs from STDIN like http://www.example.com/. Outputs paragraphs separated by newline "\n".

Ungolfed

for(; $i=parse_url(trim(fgets(STDIN))); ) {
    $h = $i['host'];
    $f = fsockopen($h, 80);
    
    fwrite($f, "GET " . $i['path'] . " HTTP/1.1\nHost:" . $h . "\nConnection:Close\n\n");
    
    $c = stream_get_contents($f)
    
    preg_match_all('!<p>(.+?)</p>!si', $c, $r);
    echo join("\n", $r[1]) . "\n";
}

First version supporting one URL only

$i=parse_url($argv[1]);fwrite($f=fsockopen($h=$i[host],80),"GET $i[path] HTTP/1.1\nHost:$h\nConnection:Close\n\n");while(!feof($f))$c.=fgets($f);preg_match_all('!<p>(.+?)</p>!sim',$c,$r);foreach($r[1]as$p)echo"$p\n";

Edits

As pointed out in the comments by Braintist, I totally forgot to include the path. That's fixed now, thanks. Added 30 bytes.
Saved 3 bytes by resetting $c (holds the page content) with $c=$i=parse_url(trim(fgets(STDIN))); instead of $c=''.
Saved 12 bytes by replacing \n with new lines (5 bytes), one while-loop with for (2 bytes), placing nearly everything into the expressions of for (2 bytes) and by replacing foreach with join (3 bytes). Thanks to Blackhole.
Saved 3 bytes by replacing fgets with stream_get_contents Thanks to bwoebi.
Saved 5 bytes by removing ~~the re-initialization of $c as it isn't needed anymore~~ $c at all.
Saved 1 byte by removing the pattern modifier m from the Regex. Thanks to manatwork

edited Jun 17, 2020 at 9:04

CommunityBot

1

answered Oct 26, 2015 at 19:14

insertusernamehere

4,7693 gold badges17 silver badges45 bronze badges

6

\$\begingroup\$ Related: stackoverflow.com/a/1732454/4766556 \$\endgroup\$
– a spaghetto
Commented Oct 26, 2015 at 19:22
1

\$\begingroup\$ @briantist Oh man, I totally missed that. :D Thanks, it's fixed now. \$\endgroup\$
– insertusernamehere
Commented Oct 26, 2015 at 19:57
1

\$\begingroup\$ I can't stand that Perl beats PHP, so don't forget: while is forbidden when golfing (for is often shorter but never longer), and to do a newline, just press enter (1 byte instead of 2 for \n)! Here is your (untested) code a bit more golfed (227 bytes), with the newline replaced by ↵: for(;$c=$i=parse_url(trim(fgets(STDIN))),fwrite($f=fsockopen($h=$i[host],80),"GET $i[path] HTTP/1.1↵Host:$h↵Connection:Close↵↵");preg_match_all('!(.+?)!sim',$c,$r),print join('↵',$r[1]).'↵')for(;!feof($f);)$c.=fgets($f); \$\endgroup\$
– Blackhole
Commented Oct 27, 2015 at 0:16
1

\$\begingroup\$ I don't mean "forbidden" as "against the rules", I just mean that's not useful at all, since a for-loop is always better than a while-loop ;). \$\endgroup\$
– Blackhole
Commented Oct 27, 2015 at 0:25
1

\$\begingroup\$ @MichaelDibbets Actually I did that already as written in the edit. Hm. Let me see. Haha, I forgot to copy and count the final snippet. Duh :D Things like that happen, if you update your code before breakfast. Thanks for pointing it out. \$\endgroup\$
– insertusernamehere
Commented Oct 27, 2015 at 10:25

| Show 12 more comments

Community · Accepted Answer · 2017-04-13 12:39:08Z

16

Perl, 132 bytes

155 bytes code + 17 for -ln -MIO::Socket - 40 for continually asking for URLs

As with @DigitalTrauma's answer, regex parsing HTML, let me know if that's not acceptable. Doesn't keep parsing URLs any more... I'll look at that later... Close to Bash though! Big thanks to @Schwern for saving me 59 (!) bytes and to @skmrx for fixing the bug to allow a claim of the bonus!

m|(http://)?([^/]+)(/(\S*))?|;$s=new IO::Socket::INET"$2:80";print$s "GET /$4 HTTP/1.1
Host:$2
Connection:close

";local$/=$,;print<$s>=~m|<p>(.+?)</p>|gs

Usage

$perl -ln -MIO::Socket -M5.010 wb.pl 
example.com
This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.<a href="http://www.iana.org/domains/example">More information...</a>
example.org
This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.<a href="http://www.iana.org/domains/example">More information...</a>

edited Apr 13, 2017 at 12:39

CommunityBot

1

answered Oct 26, 2015 at 17:47

Dom Hastings

24.3k4 gold badges55 silver badges92 bronze badges

\$\begingroup\$ I fixed a bug and shortened the code by removing the need to declare $h and $p or have a default path. It also no longer requires a trailing / on the host. \$\endgroup\$
– Schwern
Commented Oct 27, 2015 at 5:22
1

\$\begingroup\$ We're the one to beat now. :) \$\endgroup\$
– Schwern
Commented Oct 27, 2015 at 6:20
\$\begingroup\$ I think I'm done for the night. :) \$\endgroup\$
– Schwern
Commented Oct 27, 2015 at 6:54
\$\begingroup\$ Since the script asks for another URL instead of exiting, you can claim an additional -40 bytes \$\endgroup\$
– svsd
Commented Oct 27, 2015 at 13:13
1

\$\begingroup\$ @DigitalTrauma you are indeed correct! I've claimed the bonus thanks to skmrx fixing my bug with ‘$/‘ and I wouldn't be near yours if it weren't for Schwern! \$\endgroup\$
– Dom Hastings
Commented Oct 28, 2015 at 6:19

| Show 4 more comments

Community · Accepted Answer · 2017-04-13 12:39:04Z

14

PowerShell, 315 294 268 262 254 bytes

355 334 308 302 294 - 40 for prompt

$u=[uri]$args[0]
for(){
$h=$u.Host
$s=[Net.Sockets.TcpClient]::new($h,80).GetStream()
$r=[IO.StreamReader]::new($s)
$w=[IO.StreamWriter]::new($s)
$w.Write("GET $($u.PathAndQuery) HTTP/1.1
HOST: $h

")
$w.Flush()
($r.ReadToEnd()|sls '(?s)(?<=<p>).+?(?=</p>)'-a).Matches.Value
[uri]$u=Read-Host
}

Requires PowerShell v5

All line endings (including the ones embedded in the string) are newlines only \n (thanks Blackhole) which is fully supported by PowerShell (but if you're testing, be careful; ISE uses \r\n).

edited Apr 13, 2017 at 12:39

CommunityBot

1

answered Oct 26, 2015 at 19:32

briantist

3,30015 silver badges21 bronze badges

4

\$\begingroup\$ +1 for making my server admin duties appear much more productive \$\endgroup\$
– thanby
Commented Oct 27, 2015 at 11:20
\$\begingroup\$ HTTP requires CRLF, not LF! [HTTPSYNTAX] \$\endgroup\$
– Toothbrush
Commented Oct 29, 2015 at 19:40
2

\$\begingroup\$ @toothbrush Ha! Point taken, but the tolerance provision seems to be in full effect. Clearly this task is about what works and not what's correct (otherwise we wouldn't be parsing HTML with regex and using low level TCP libraries instead of well-tested existing libraries). \$\endgroup\$
– briantist
Commented Oct 29, 2015 at 19:44
1

\$\begingroup\$ @briantist greenbytes.de/tech/webdav/rfc7230.html#rfc.section.3.5 says that "a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR". I read that as meaning most web servers would implement it, and the question definitely doesn't say it must generate correct GET requests… :) \$\endgroup\$
– Toothbrush
Commented Oct 29, 2015 at 19:52

Add a comment |

philcolbourn · Accepted Answer · 2015-10-28 11:25:16Z

9

Bash (might be cheating but seems to be within rules) 144-40=105

while read a;do
u=${a#*//}
d=${u%%/*}
e=www.w3.org
exec 3<>/dev/tcp/$d/80
echo "GET /services/html2txt?url=$a HTTP/1.1
Host:$d
">&3
cat <&3
done

Thanks to Digital Trauma.

Since I don't need to split URL, this also works: 122-40=82

while read a;do
d=www.w3.org
exec 3<>/dev/tcp/$d/80
echo "GET /services/html2txt?url=$a HTTP/1.1
Host:$d
">&3   
cat <&3
done

edited Oct 28, 2015 at 11:25

answered Oct 28, 2015 at 11:12

philcolbourn

5613 silver badges8 bronze badges

9

\$\begingroup\$ I would argue that using this online html2txt converter is a standard loophole \$\endgroup\$
– Digital Trauma
Commented Oct 28, 2015 at 16:58
1

\$\begingroup\$ Yes. And I also use cat so your solution is safe. \$\endgroup\$
– philcolbourn
Commented Oct 29, 2015 at 0:06

Add a comment |

Rnet · Accepted Answer · 2015-10-28 13:08:51Z

9

Groovy script, 89, 61 bytes

Loop back for bonus 101- 40 = 61

System.in.eachLine{l->l.toURL().text.findAll(/<p>(?s)(.*?)<\/p>/).each{println it[3..it.length()-5]}}

With just args, 89 bytes

this.args[0].toURL().text.findAll(/<p>(?s)(.*?)<\/p>/).each{println it[3..it.length()-5]}

edited Oct 28, 2015 at 13:08

answered Oct 28, 2015 at 13:01

Rnet

2971 silver badge5 bronze badges

1

\$\begingroup\$ Groovy outgolfed everyone. As it should be. \$\endgroup\$
– a spaghetto
Commented Oct 28, 2015 at 19:09
1

\$\begingroup\$ @quartata If it stays that way, it'll be the first time ever, so... ;) \$\endgroup\$
– Geobits
Commented Oct 28, 2015 at 19:11
11

\$\begingroup\$ "only low-level TCP APIs are allowed" \$\endgroup\$
– Digital Trauma
Commented Oct 28, 2015 at 21:19
\$\begingroup\$ Yeah i'm going to agree with @DigitalTrauma that this isn't using a low-level TCP API. The rules state that you have to split the host and path on your own. \$\endgroup\$
– TheDoctor
Commented Nov 8, 2015 at 15:37

Add a comment |

ezrast · Accepted Answer · 2015-10-29 23:22:14Z

Ruby, 118

147 bytes source; 11 bytes '-lprsocket'; -40 bytes for looping.

*_,h,p=$_.split'/',4
$_=(TCPSocket.new(h,80)<<"GET /#{p} HTTP/1.1
Host:#{h}
Connection:close

").read.gsub(/((\A|<\/p>).*?)?(<p>|\Z)/mi,'
').strip

Usage example:

$ ruby -lprsocket wb.rb
http://example.org/
This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.
<a href="http://www.iana.org/domains/example">More information...</a>
http://www.xkcd.com/1596/
Warning: this comic occasionally contains strong language (which may be unsuitable for children), unusual humor (which may be unsuitable for adults), and advanced mathematics (which may be unsuitable for liberal-arts majors).

This work is licensed under a
<a href="http://creativecommons.org/licenses/by-nc/2.5/">Creative Commons Attribution-NonCommercial 2.5 License</a>.


This means you're free to copy and share these comics (but not to sell them). <a rel="license" href="/license.html">More details</a>.

Community · Accepted Answer · 2017-04-13 12:39:03Z

6

C 512 Bytes

#include <netdb.h>
int main(){char i,S[999],b[99],*p,s=socket(2,1,0),*m[]={"<p>","</p>"};long n;
gets(S);p=strchr(S,'/');*p++=0;struct sockaddr_in a={0,2,5<<12};memcpy(&a.
sin_addr,gethostbyname(S)->h_addr,4);connect(s,&a,16);send(s,b,sprintf(b,
"GET /%s HTTP/1.0\r\nHost:%s\r\nAccept:*/*\r\nConnection:close\r\n\r\n",p,S),0);
p=m[i=0];while((n=recv(s,b,98,0))>0)for(char*c=b;c<b+n;c++){while(*c==*p &&*++p)
c++;if(!*p)p=m[(i=!i)||puts("")];else{while(p>m[i]){if(i)putchar(c[m[i]-p]);p--;}
if(i)putchar(*c);}}}

Based loosely on my entry here, It takes the web address without a leading "https://". It will not handle nested  pairs correctly :(

Tested extensively on www.w3.org/People/Berners-Lee/
It works when compiled with Apple LLVM version 6.1.0 (clang-602.0.53) / Target: x86_64-apple-darwin14.1.1
It has enough undefined behavior that it may not work anywhere else.

edited Apr 13, 2017 at 12:39

CommunityBot

1

answered Oct 28, 2015 at 22:13

AShelly

4,6511 gold badge22 silver badges37 bronze badges

\$\begingroup\$ I was going down roughly the same track (this segfaults when compiled with gcc), but it should be possible to get under 400 bytes in C. Not sure about clang, but you shouldn't have to declare the return type of main. You can also remove the include and "access" the structs as integer arrays instead. I've also been getting responses with "GET /%s HTTP/1.1\r\n\r\n\", but mileage on that may vary based on the site... \$\endgroup\$
– Comintern
Commented Oct 29, 2015 at 2:19
\$\begingroup\$ 483 bytes \$\endgroup\$
– ceilingcat
Commented Apr 29, 2021 at 3:29

Add a comment |

user42643user42643 · Accepted Answer · 2015-10-26 17:52:04Z

AutoIt, 347 bytes

Func _($0)
$4=StringTrimLeft
$0=$4($0,7)
$3=StringSplit($0,"/")[1]
TCPStartup()
$2=TCPConnect(TCPNameToIP($3),80)
TCPSend($2,'GET /'&$4($0,StringLen($3))&' HTTP/1.1'&@LF&'Host: '&$3&@LF&'Connection: close'&@LF&@LF)
$1=''
Do
$1&=TCPRecv($2,1)
Until @extended
For $5 In StringRegExp($1,"(?s)\Q<p>\E(.*?)(?=\Q</p>\E)",3)
ConsoleWrite($5)
Next
EndFunc

Testing

Input:

_('http://www.autoitscript.com')

Output:

You don't have permission to access /error/noindex.html
on this server.

Input:

_('http://www.autoitscript.com/site')

Output:

The document has moved <a href="https://www.autoitscript.com/site">here</a>.

Remarks

Doesn't support nested  tags
Supports only  tags (case-insensitive), will break on every other tag format
~~Panics~~ Loops indefinitely when any error occurs

Stephan Schinkel · Accepted Answer · 2015-10-28 16:10:27Z

C#, 727 Bytes - 40 = 687 Bytes

using System.Text.RegularExpressions;class P{static void Main(){a:var i=System.Console.ReadLine();if(i.StartsWith("http://"))i=i.Substring(7);string p="/",h=i;var l=i.IndexOf(p);
if(l>0){h=i.Substring(0,l);p=i.Substring(l,i.Length-l);}var c=new System.Net.Sockets.TcpClient(h,80);var e=System.Text.Encoding.ASCII;var d=e.GetBytes("GET "+p+@" HTTP/1.1
Host: "+h+@"
Connection: close

");var s=c.GetStream();s.Write(d,0,d.Length);byte[]b=new byte[256],o;var m=new System.IO.MemoryStream();while(true){var r=s.Read(b,0,b.Length);if(r<=0){o=m.ToArray();break;}m.Write(b,0,r);}foreach (Match x in new Regex("<p>(.+?)</p>",RegexOptions.Singleline).Matches(e.GetString(o)))System.Console.WriteLine(x.Groups[1].Value);goto a;}}

It's a little bit of training but surely memorable :)

Here is an ungolfed version:

using System.Text.RegularExpressions;
class P
{
    static void Main()
    {
    a:
        var input = System.Console.ReadLine();
        if (input.StartsWith("http://")) input = input.Substring(7);
        string path = "/", hostname = input;
        var firstSlashIndex = input.IndexOf(path);
        if (firstSlashIndex > 0)
        {
            hostname = input.Substring(0, firstSlashIndex);
            path = input.Substring(firstSlashIndex, input.Length - firstSlashIndex);
        }
        var tcpClient = new System.Net.Sockets.TcpClient(hostname, 80);
        var asciiEncoding = System.Text.Encoding.ASCII;
        var dataToSend = asciiEncoding.GetBytes("GET " + path + @" HTTP/1.1
Host: " + hostname + @"
Connection: close

");
        var stream = tcpClient.GetStream();
        stream.Write(dataToSend, 0, dataToSend.Length);
        byte[] buff = new byte[256], output;
        var ms = new System.IO.MemoryStream();
        while (true)
        {
            var numberOfBytesRead = stream.Read(buff, 0, buff.Length);
            if (numberOfBytesRead <= 0)
            {
                output = ms.ToArray();
                break;
            }
            ms.Write(buff, 0, numberOfBytesRead);
        }
        foreach (Match match in new Regex("<p>(.+?)</p>", RegexOptions.Singleline).Matches(asciiEncoding.GetString(output)))
        {
            System.Console.WriteLine(match.Groups[1].Value);
            goto a;
        }
    }
}

As you can see, there are memory leak issues as a bonus :)

Where is the memory leak? I see no using statements around streams but that does not make a leak. — Gusdor, Commented Oct 29, 2015 at 14:41
You can trim a few more bytes: input = input.trimStart("http://") will replace the "if" clause, and you should be able to use System.Text.Encoding.ASCII.GetBytes() directly without having to store it in asciiEncoding first. I think you'd even come out ahead with a "Using System;" line and getting rid of a handful of "System."s. — minnmass, Commented Oct 30, 2015 at 5:04

Community · Accepted Answer · 2020-06-17 09:04:33Z

4

JavaScript (NodeJS) - 187 166

s=require("net").connect(80,p=process.argv[2],_=>s.write("GET / HTTP/1.0\nHost: "+p+"\n\n")&s.on("data",d=>(d+"").replace(/<p>([^]+?)<\/p>/g,(_,g)=>console.log(g))));

187:

s=require("net").connect(80,p=process.argv[2],_=>s.write("GET / HTTP/1.1\nHost: "+p+"\nConnection: close\n\n")&s.on("data",d=>(d+"").replace(/<p>([^]+?)<\/p>/gm,(_,g)=>console.log(g))));

Usage:

node file.js www.example.com

Or formatted

var url = process.argv[2];
s=require("net").connect(80, url ,_=> {
     s.write("GET / HTTP/1.1\nHost: "+url+"\nConnection: close\n\n");
     s.on("data",d=>(d+"").replace(/<p>([^]+?)<\/p>/gm,(_,g)=>console.log(g)))
});

edited Jun 17, 2020 at 9:04

CommunityBot

1

answered Oct 28, 2015 at 9:01

Benjamin Gruenbaum

2291 silver badge9 bronze badges

1

\$\begingroup\$ Caveat: this will work for small pages - bigger pages emit multiple data events. \$\endgroup\$
– Benjamin Gruenbaum
Commented Oct 28, 2015 at 9:04

Add a comment |

Zac Crites · Accepted Answer · 2015-10-28 18:20:41Z

3

Python 2 - 212 209 bytes

import socket,re
h,_,d=raw_input().partition('/')
s=socket.create_connection((h,80))
s.sendall('GET /%s HTTP/1.1\nHost:%s\n\n'%(d,h))
p=''
while h:h=s.recv(9);p+=h
for g in re.findall('<p>(.*?)</p>',p):print g

edited Oct 28, 2015 at 18:20

answered Oct 28, 2015 at 13:45

Zac Crites

2011 silver badge4 bronze badges

\$\begingroup\$ You can save two bytes by stripping out whitespace after the colon on while h: and before print g. \$\endgroup\$
– Skyler
Commented Oct 28, 2015 at 16:25
\$\begingroup\$ And another byte with 'GET /%s HTTP/1.1\nHost:%s\n\n'. \$\endgroup\$
– Cees Timmerman
Commented Oct 28, 2015 at 16:48

Add a comment |

Community · Accepted Answer · 2017-04-13 12:39:03Z

Python 2, 187 - 40 = 147 (141 in a REPL)

Compressed and looped version of Zac's answer:

import socket,re
while 1:h,_,d=raw_input().partition('/');s=socket.create_connection((h,80));s.sendall('GET /%s HTTP/1.1\nHost:%s\n\n'%(d,h));print re.findall('<p>(.*?)</p>',s.recv(9000))

Example:

dictionary.com
['The document has moved <a href="http://dictionary.reference.com/">here</a>.']
dictionary.reference.com
[]
paragraph.com
[]
rare.com
[]

Actually useful is this:

207 - 40 = 167

import socket,re
while 1:h,_,d=raw_input().partition('/');s=socket.create_connection((h,80));s.sendall('GET /%s HTTP/1.1\nHost:%s\n\n'%(d,h));print'\n'.join(re.findall('<p>(.*?)</p>',s.recv(9000),re.DOTALL))

Example:

example.org
This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.
<a href="http://www.iana.org/domains/example">More information...</a>
www.iana.org/domains/example
The document has moved <a href="/domains/reserved">here</a>.
www.iana.org/domains/reserved

dictionary.com
The document has moved <a href="http://dictionary.reference.com/">here</a>.
dictionary.reference.com

catb.org

      <a href="http://validator.w3.org/check/referer"><img
          src="http://www.w3.org/Icons/valid-xhtml10"
          alt="Valid XHTML 1.0!" height="31" width="88" /></a>

This is catb.org, named after (the) Cathedral and the Bazaar. Most
of it, under directory esr, is my personal site.  In theory other
people could shelter here as well, but this has yet to occur.
catb.org/jargon
The document has moved <a href="http://www.catb.org/jargon/">here</a>.
www.catb.org/jargon/
This page indexes all the WWW resources associated with the Jargon File
and its print version, <cite>The New Hacker's Dictionary</cite>. It's as
official as anything associated with the Jargon File gets.
On 23 October 2003, the Jargon File achieved the
dubious honor of being cited in the SCO-vs.-IBM lawsuit.  See the <a
href='html/F/FUD.html'>FUD</a> entry for details.
www.catb.org/jargon/html/F/FUD.html
 Defined by Gene Amdahl after he left IBM to found his own company:
   &#8220;<span class="quote">FUD is the fear, uncertainty, and doubt that IBM sales people
   instill in the minds of potential customers who might be considering
   [Amdahl] products.</span>&#8221; The idea, of course, was to persuade them to go
   with safe IBM gear rather than with competitors' equipment.  This implicit
   coercion was traditionally accomplished by promising that Good Things would
   happen to people who stuck with IBM, but Dark Shadows loomed over the
   future of competitors' equipment or software.  See
   <a href="../I/IBM.html"><i class="glossterm">IBM</i></a>.  After 1990 the term FUD was associated
   increasingly frequently with <a href="../M/Microsoft.html"><i class="glossterm">Microsoft</i></a>, and has
   become generalized to refer to any kind of disinformation used as a
   competitive weapon.
[In 2003, SCO sued IBM in an action which, among other things,
   alleged SCO's proprietary control of <a href="../L/Linux.html"><i class="glossterm">Linux</i></a>.  The SCO
   suit rapidly became infamous for the number and magnitude of falsehoods
   alleged in SCO's filings.  In October 2003, SCO's lawyers filed a <a href="http://www.groklaw.net/article.php?story=20031024191141102" target="_top">memorandum</a>
   in which they actually had the temerity to link to the web version of
   <span class="emphasis"><em>this entry</em></span> in furtherance of their claims. Whilst we
   appreciate the compliment of being treated as an authority, we can return
   it only by observing that SCO has become a nest of liars and thieves
   compared to which IBM at its historic worst looked positively
   angelic. Any judge or law clerk reading this should surf through to
   <a href="http://www.catb.org/~esr/sco.html" target="_top">my collected resources</a> on this
   topic for the appalling details.&#8212;ESR]

Cabbie407 · Accepted Answer · 2015-10-27 18:16:06Z

gawk, 235 - 40 = 195 bytes

{for(print"GET "substr($0,j)" HTTP/1.1\nHost:"h"\n"|&(x="/inet/tcp/0/"(h=substr($0,1,(j=index($0,"/"))-1))"/80");(x|&getline)>0;)w=w RS$0
for(;o=index(w,"<p>");w=substr(w,c))print substr(w=substr(w,o+3),1,c=index(w,"/p>")-2)
close(x)}

Golfed it down, but this is a more unforgiving version, which needs the web address without http:// at the beginning. And if you want to access the root directory you have to end the address with a /. Furthermore the  tags have to be lower case.

My earlier version actually didn't handle lines containing  correctly. This is now fixed.

Output for input `example.com/`

This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.
<a href="http://www.iana.org/domains/example">More information...</a>

Still doesn't work with Wikipedia. I think the reason is that Wikipedia uses https for everything. But I don't know.

The following version is a little more forgiving with the input and it can handle upper case tags as well.

IGNORECASE=1{
    s=substr($0,(i=index($0,"//"))?i+2:0)
    x="/inet/tcp/0/"(h=(j=index(s,"/"))?substr(s,1,j-1):s)"/80"
    print"GET "substr(s,j)" HTTP/1.1\nHost:"h"\nConnection:close\n"|&x
    while((x|&getline)>0)w=w RS$0
    for(;o=index(w,"<p>");w=substr(w,c))
        print substr(w=substr(w,o+3),1,c=index(w,"/p>")-2)
    close(x)
}

I'm not sure about the "Connection:close" line. Doesn't seem to be mandatory. I couldn't find an example that would work different with or without it.

dwana · Accepted Answer · 2015-11-06 12:14:18Z

Powershell(4) 240

$input=Read-Host ""
$url=[uri]$input
$dir=$url.LocalPath
Do{
$res=Invoke-WebRequest -URI($url.Host+"/"+$dir) -Method Get
$res.ParsedHtml.getElementsByTagName('p')|foreach-object{write-host $_.innerText}
$dir=Read-Host ""
}While($dir -NE "")

Ungolfed (proxy is not required)

$system_proxyUri=Get-ItemProperty -Path "HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings" -Name ProxyServer
$proxy = [System.Net.WebRequest]::GetSystemWebProxy()
$proxyUri = $proxy.GetProxy($system_proxyUri.ProxyServer)
$input = Read-Host "Initial url"
#$input="http://stackoverflow.com/questions/tagged/powershell"
$url=[uri]$input
$dir=$url.LocalPath
Do{
$res=Invoke-WebRequest -URI($url.Host+"/"+$dir) -Method Get -Proxy($proxyUri)
$res.ParsedHtml.getElementsByTagName('p')|foreach-object{write-host $_.innerText}
$dir=Read-Host "next dir"
}While($dir -NE "")

edit* also not to hard to memorize ^^

sech1p · Accepted Answer · 2021-04-25 00:35:45Z

Java (OpenJDK 15), 790 bytes - 40 bytes = 750 bytes

import java.io.*;import java.net.*;import java.util.*;import java.util.regex.*;interface A{static void main(String[]a)throws IOException{Pattern p=Pattern.compile("<p(.*?)>(.*?)</p>",Pattern.DOTALL);Matcher m;while(true){URL u=new URL(new Scanner(System.in).nextLine());Socket s=new Socket(u.getHost(),80);PrintStream ps=new PrintStream(s.getOutputStream());BufferedReader b=new BufferedReader(new InputStreamReader(s.getInputStream()));ps.print("GET "+(u.getPath().length()==0?"/":u.getPath())+" HTTP/1.1\r\nHost: "+u.getHost()+"\r\nConnection: close\r\n\r\n");var l=b.readLine();var r=new StringBuilder();while(l!=null){l=b.readLine();r.append(l).append("\n");}m=p.matcher(r);while(m.find()){System.out.println(m.group().replaceAll("<p(.*?)>|</p>",""));}ps.close();b.close();s.close();}}}

Ungolfed version:

package rocks.sech1p;

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.regex.*;

public class Main {
    public static void main(String[] args) throws IOException {
        Scanner input = new Scanner(System.in);
        Pattern pattern = Pattern.compile("<p(.*?)>(.*?)</p>", Pattern.DOTALL);
        Matcher match;

        while(true) {
            URL url = new URL(input.nextLine());
            Socket socket = new Socket(url.getHost(), 80);
            PrintStream printstream = new PrintStream(socket.getOutputStream());
            BufferedReader buffer = new BufferedReader(new InputStreamReader(socket.getInputStream()));

            printstream.print("GET " + (url.getPath().length() == 0 ? "/" : url.getPath()) + " HTTP/1.1\r\nHost: " + url.getHost() + "\r\nConnection: close\r\n\r\n");

            String line = buffer.readLine();
            StringBuilder response = new StringBuilder();
            while (line != null) {
                line = buffer.readLine();
                response.append(line).append("\n");
            }
            match = pattern.matcher(response);
            while(match.find())
                System.out.println(match.group().replaceAll("<p(.*?)>|</p>", ""));

            printstream.close();
            buffer.close();
            socket.close();
        }
    }
}

\$\begingroup\$ 649 bytes \$\endgroup\$
– ceilingcat
Commented Apr 27, 2021 at 4:16 — ceilingcat, Commented Apr 27, 2021 at 4:16

Stack Exchange Network

The World's Smallest Web Browser

16 Answers 16

Pure Bash (no utilities), 200 bytes - 40 bonus = 160

PHP, 175 bytes (215 - 40 bonus) 227 229 239 202 216 186 bytes

Perl, 132 bytes

Usage

PowerShell, 315 294 268 262 254 bytes

355 334 308 302 294 - 40 for prompt

Groovy script, 89, 61 bytes

Ruby, 118

Usage example:

C 512 Bytes

AutoIt, 347 bytes

Testing

Remarks

C#, 727 Bytes - 40 = 687 Bytes

JavaScript (NodeJS) - 187 166

Python 2 - 212 209 bytes

Python 2, 187 - 40 = 147 (141 in a REPL)

207 - 40 = 167

gawk, 235 - 40 = 195 bytes

Output for input `example.com/`

Java (OpenJDK 15), 790 bytes - 40 bytes = 750 bytes

Your Answer

Not the answer you're looking for? Browse other questions tagged
code-golf
parsing
internet
or ask your own question.

Linked

Hot Network Questions

The World's Smallest Web Browser

16 Answers 16

Pure Bash (no utilities), 200 bytes - 40 bonus = 160

PHP, 175 bytes (215 - 40 bonus) 227 229 239 202 216 186 bytes

Perl, 132 bytes

Usage

PowerShell, 315 294 268 262 254 bytes

355 334 308 302 294 - 40 for prompt

Groovy script, 89, 61 bytes

Ruby, 118

Usage example:

C 512 Bytes

AutoIt, 347 bytes

Testing

Remarks

C#, 727 Bytes - 40 = 687 Bytes

JavaScript (NodeJS) - 187 166

Python 2 - 212 209 bytes

Python 2, 187 - 40 = 147 (141 in a REPL)

207 - 40 = 167

gawk, 235 - 40 = 195 bytes

Output for input example.com/

Java (OpenJDK 15), 790 bytes - 40 bytes = 750 bytes

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged code-golfparsinginternet or ask your own question.

Linked

Related

Hot Network Questions

Output for input `example.com/`

Not the answer you're looking for? Browse other questions tagged
code-golf
parsing
internet
or ask your own question.