User:Matt Crypto/RandomArticles
Appearance
While Wikipedia has a Random page feature, the pages are selected uniformly randomly from the database. As an alternative, I wrote a script to choose pages randomly based on their hit counts for a month; such a set might give a more representative example of how Wikipedia looks to visitors. The hit data for, say, September 2004 can be found here (warning: very large file). Below is an example from the hits so far this month (to 22nd September 2004). If you would like a set, just send me a message and tell me a Wikipedia page, and I'll run the script for you and paste in the output. — Matt 15:06, 21 Sep 2004 (UTC)
100 randomly-selected articles (weighted by popularity)
[edit]- Embryophyte — (51 hits)
- CRAF — (36 hits)
- Congressional committee — (35 hits)
- Snake — (870 hits)
- Linux distribution — (687 hits)
- Plate Carrée Projection — (20 hits)
- Place Manner Time — (77 hits)
- Sestertius — (124 hits)
- Stargate SG-1 — (661 hits)
- Moorhead, Minnesota — (24 hits)
- MOHAA — (26 hits)
- Ian Stuart — (6 hits)
- Readelf — (42 hits)
- Sidney James — (38 hits)
- Jacques Derrida — (808 hits)
- Edgar Degas — (1668 hits)
- Strategic bombing — (270 hits)
- The Kingston Trio — (77 hits)
- Zoophilia — (11612 hits)
- United States Senate — (2472 hits)
- Women's Social and Political Union — (73 hits)
- Prostaglandin — (596 hits)
- Painters — (208 hits)
- Archeology of Algeria — (16 hits)
- Nyota — (10 hits)
- Nikkei Index — (17 hits)
- Norway — (2809 hits)
- Coefficient — (86 hits)
- Chinese mantis — (46 hits)
- Triple — (107 hits)
- Minor characters from The Hitchhiker's Guide to the Galaxy — (780 hits)
- History of Seattle — (79 hits)
- Dawes Rolls — (66 hits)
- John Stewart — (40 hits)
- Puberty — (1573 hits)
- Electrical resistance — (806 hits)
- Sophia — (225 hits)
- Hydroponic (album) — (5 hits)
- Biafran War — (218 hits)
- Halloween documents — (170 hits)
- Squad Automatic Weapon — (47 hits)
- Carl Wayne — (210 hits)
- British Forces Germany — (53 hits)
- Beslan hostage crisis — (14675 hits)
- Craigieburn — (12 hits)
- Spot (Star Trek) — (131 hits)
- Smart (automobile) — (941 hits)
- Microscope — (3498 hits)
- Time value of money — (117 hits)
- George Jackson — (50 hits)
- Clarence — (21 hits)
- Communication with submarines — (789 hits)
- Macaulay Culkin — (597 hits)
- Jade Emperor — (194 hits)
- Jimbo Wales — (514 hits)
- Round Table — (146 hits)
- Arizona State University — (606 hits)
- List of regions of the United States — (1173 hits)
- King's College, Cambridge — (165 hits)
- Rhythmic gesture — (17 hits)
- Longest word in English — (1405 hits)
- Condorcet method — (806 hits)
- Total Recall — (214 hits)
- Shawn Michaels — (334 hits)
- Conjunction fallacy — (142 hits)
- 2004 Summer Olympics medal count — (1747 hits)
- Pizza — (695 hits)
- Ambisonics — (4 hits)
- Paul Neil Milne Johnstone — (212 hits)
- HMS Albion (1802) — (4 hits)
- Contagious magic — (5 hits)
- Phase velocity — (124 hits)
- IWW — (120 hits)
- Vegetarian — (355 hits)
- Schlong — (26 hits)
- Auschwitz Album — (3970 hits)
- GameFAQs — (1317 hits)
- Meteorology — (554 hits)
- Connotation — (537 hits)
- Oral sex — (7430 hits)
- 1969 — (1749 hits)
- Nucleic acid — (452 hits)
- Alcohol — (1846 hits)
- Uluru — (376 hits)
- EMac — (136 hits)
- Montagu Island — (30 hits)
- Black Panther — (153 hits)
- Orlando Letelier — (192 hits)
- Godwin's law — (6776 hits)
- Tybee Bomb — (2609 hits)
- Spaced — (78 hits)
- BAC 1-11 — (61 hits)
- 1974 in film — (234 hits)
- Relational model — (609 hits)
- Property — (508 hits)
- Glasgow — (704 hits)
- Nicotine — (408 hits)
- Rear Window — (177 hits)
- Texas Air National Guard controversy — (166 hits)
- Football World Cup 1974 — (85 hits)
Script
[edit]import re from random import * logFile = "/tmp/url_200409.html" maxEntries = None # 10000 numberOfArticles = 100 r1 = re.compile(r'^(\d*)\s*([0-9.]*)%\s*([0-9]*)\s*([0-9.]*)%\s*/wiki/(\S*)$') class ArticlePicker: def __init__(self, logFile, maxEntries = False): self.logFile = logFile self.hitList = [] self.count = 0 self.maxEntries = maxEntries def readLogFile(self): F = open(self.logFile) count = 0 self.hitSum = 0 for l in F: if self.maxEntries and count > self.maxEntries: break try: hits, name = self.parseLine(l) except ValueError: continue count = count + 1 self.hitList.append((hits,name)) self.hitSum += hits self.count = count F.close() self.hitList.sort() self.hitList.reverse() def parseLine(self, line): l = line.strip() m = r1.match(l) if m == None: raise ValueError, "No matches found" (hits, t1, t2, t3, name) = r1.match(l).groups() self.filterOut(hits, name) spaceName = re.sub('_', ' ', name) return int(hits), spaceName def filterOut(self, hits, name): if name == "": raise ValueError # Exclude blank if re.match(r'^\w*:', name): raise ValueError # Exclude namespaces if re.match(r'Main_Page', name): raise ValueError # Exclude main page # Exclude popular oddities if re.match(r'_vti_bin/owssvr.dl|MSOffice/cltreq.asp', name): raise ValueError def selectRandomly(self, N = 1): rHits = [random() * self.hitSum for i in range(N)] outputs = [None] * N numberOfOutputs = 0 totalSoFar = 0 for hits, name in self.hitList: totalSoFar += hits for index in range(N): if not outputs[index] and totalSoFar >= rHits[index]: outputs[index] = hits, name numberOfOutputs += 1 if numberOfOutputs == N: return outputs return outputs # Dump the articles H = ArticlePicker(logFile, maxEntries) H.readLogFile() randomArticles = H.selectRandomly(numberOfArticles) print "==%d randomly-selected articles (weighted by popularity)==" % numberOfArticles for hits, name in randomArticles: print "* %s — (%d hits)" % (name, hits)