Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Are HN submissions dominated by a small elite of users?
36 points by jhh on Dec 27, 2013 | hide | past | favorite | 25 comments
I was wondering the same thing when I recently saw this blog post: https://news.ycombinator.com/item?id=6956690

I thought it would be easy to find out how karma is distributed among users using the HNsearch API.

Unfortunately, it's not that easy because this API is not optimized for fetching massive amounts of data. What I could do was looking at the 1000 users with the most karma. I separated those into ten chunks and looked at the total sum of Karma per chunk of 100 users. This is the result:

http://www.chartgo.com/share.do?id=6cca3d09aa

(Data: [ (0, 2941707), (1, 1314760), (2, 936226), (3, 745944), (4, 636533), (5, 557441), (6, 493996), (7, 449999), (8, 413963), (9, 384491)] )

So as you can see the 1st chunk is pretty dominant but then it kind of flattens out. So are HN submissions dominated by a small elite? It obviously depends on your definitions, but I guess not.

It would be interesting to have better data available. Did I miss a better way to query the API? Here's the Python script which I used to query the API: https://gist.github.com/johannes-gehrs/77a92284e3509f2ae960



If you're talking about submission karma, clearly no. I follow the submissions of a selection of HN users I've known for awhile, all of whom are karma leaders, and their submissions are often as not buried on page 4 or 5 with 2-3 votes.

If you're talking about comment karma, clearly yes. I could write a comment consisting of randomly chosen names from the Chicago phone book and someone would upvote it. Name recognition has that effect, as does a small group of people who seem to follow my comments deliberately (as I do for the list of people in my profile).


Not just name recognition, but a tptacek/pg/patio11 comment is landed #1 on the comment page and stays there a long time just because of karma. Then, occupying the most privileged real state on the page, after thousands of views, even if it is a small percentage that likes the comment, it is a lot of upvotes.

Not that I disagree with this algorithm, I actually agree. Just saying that I think the position of the comment is more important than name recognition in this case.


grellas posts are particularly buried because they often don't subscribe to the hivemind, but they are always enlightening.


'grellas comments are (justifiably) routinely stapled to the top of the threads on which they appear.


Keep in mind that karma includes both submissions and comments (unlike reddit, where they are separate).

For comments, at least, there is a clear bias where comments from users with higher karma tend to stay at the top of the comment page longer, regardless of the votes for other comments. This is intentional and definitely contributes to the effect


Sort of a "founder effect" built into a site for founders. How meta.


I suspect that you'll find it to be some kind of power law distribution, which implies that measures like Standard Deviation won't be so useful. A quick look at the top 1000 that you provided suggests this might be the case. But the top 100 users don't dominate the top 1000, let alone the entire population. There does seem to be some leveling off, though it's unclear how steady that will be.

Net - domination is too strong a word. Dominated might imply 20% of the karma in 20 users, and this clearly isn't the case. Even defining the Elite as 100 doesn't get us this.


Yes, take a look at the "new" page. On it you'll see 30 recent submissions, most of which are just as appropriate as those on the home page. But, they don't get any votes, why not?

I don't think it typically matters whether the submitter has name recognition or karma, but rather how many friends and colleagues the person can rally to their cause. All it takes is about 10 points in the first hour to get to the front page and then it can "ride the wave" from there. This is an easy threshold for people with large professional networks, companies, with friends in SV et al. Many people here know each other. Yes, there are some technical counter-measures, but there is only so much that can be done.

This is not to say that a user acting alone has no chance of reaching the home page. However, after a few years of frequenting this site, I believe that most of the posts hitting the front page have had a boost such as I described above.

My personal experience anecdote is that, over several years (also considering prior accounts), working as a "loner", that I've never been able to get more than a few meager votes on a submission. Even if they were highly on topic, even if I got up at 6am Pacific, supposedly the best time to do it. Maybe I'm unlucky or my submissions suck, but I rarely bother any more.


My impression is that suppression of voting rings, which is what you're describing with your "group of 10 friends" example, is something that the HN maintainers actually spend a lot of effort on; I get this both firsthand from the maintainers that I actually know, and from the fact that I feel like I regularly manage to do (innocuous) things that trip the detector.


HN voting is a trivial example of selection bias. If frontpage was determined by a random sampling of the user base (weighted by karma or age if you like), instead of whoever may go looking for the post, it would be more fair. One of the reddit devs suggested something like this recently, putting a New post on every page.


They do, but avoiding the counter-measures isn't that hard if you're aware of them, and importantly, only needs to be done a few times a year for each combination of friends.

I could be mistaken, but the hypothesis fits so far.


I have no friends on HN and I tell no one when I'm submitting something and yet I've managed to post things that get to the front page fairly regularly.


Looks like you have a roughly ten percent success rate. Sounds reasonable.



A little late to this, but for my Hacker Newsletter project I keep track of of the number of times a user gets on the top three pages:

  337 shawndumas
  312 ColinWright
  298 iProject
  291 Libertatea
  214 danso
  197 llambda
  165 Lightning
  163 tokenadult
  158 sethbannon
  147 mtgx
  144 evo_9
  120 DanielRibeiro
  115 denzil_correa, jonbaer
  111 lelf
  102 001sky
  101 dmor
  98 coloneltcb
  97 Tsiolkovsky, craigkerstiens, dsr12, mh_
  95 ssclafani
  90 jseliger
  89 jamesbritt
  87 JumpCrisscross
  85 Garbage
  83 wallflower
  82 luu
  81 antr, protomyth
  77 anigbrowl, codegeek
  74 aaronbrethorst
  73 jgrahamc, pg
  71 uladzislau
  70 sinak, steveklabnik
  69 RougeFemme
  68 cleverjake, derpenxyne, swohns
  67 _pius
  66 jamesjyu, rosser, wslh
  64 Pr0, eplanit
  63 Brajeshwar, sk2code, uptown
  62 bane, stfu
  61 teawithcarl
  59 ghosh, joeyespo, ph0rque
  58 gnosis
  57 r0h1n, scholia
  56 AndrewDucker, cpeterso, ohjeez
  55 bpierre, cyphersanctus, gruseom
  54 conductor
  52 nikunjk
  51 Suraj-Sun, kirillzubovsky
  50 lispython
  49 MikeCapone, WestCoastJustin, co_pl_te, duck, robin_reala
  48 Anon84, ca98am79, taylorbuley
  47 fraqed, morphics, nkurz, rpm4321
  46 jmduke, kumarski, nreece
  45 georgecmu
  44 daegloe, rdl
  43 donohoe, kunle, olivercameron
  42 tanglesome
  41 austengary, ck2, jnazario
  40 interconnector, rnyman, sonabinu, wikiburner, zoowar
  39 cdvonstinkpot, pwg, revorad
  38 1337biz, hepha1979, jeffbarr, mhb, zdw
  37 fogus, kjhughes, vectorbunny
  36 Cbasedlifeform, DanBC, DiabloD3, Reltair, acremades, geetarista
  35 AndreyKarpov, jessaustin, mxfh, prostoalex, raganwald, rbanffy
  34 wglb
  33 CrankyBear, cryptoz, hackhackhack, kposehn, nvk
  32 MarlonPro, bcn, npguy, qubitsam, techinsidr
  31 Ashuu, Baustin, caffeinewriter, drucken, edent, iamtechaddict, ilamont, mariuz, mikecane, rmah, rohshall, shill, whoishiring
  30 eguizzo, eksith, gklein, lukashed, talhof8
  29 Sami_Lehtinen, ari_elle, cwan, espeed, gwern, jacquesm, mitmads, sgdesign, superchink, velodrome, wyclif
  28 Quekster, X4, clicks, esalazar, iamwil, jaf12duke, javinpaul, laurent123456, recoiledsnake, vanwilder77, zt
  27 6thSigma, ValentineC, aespinoza, ananyob, anu_gupta, awwstn, bavidar, dave1010uk, ekm2, pajju, rkudeshi, sdoering, yapcguy
  26 DanielBMarkham, bitsweet, brudgers, hornokplease, jcr, tptacek, whalesalad
  25 EzGraphs, Hirvesh, adventured, albertzeyer, barredo, johns, jpadilla_, kevin_morrill, kmfrk, mmastrac, mooreds, mrb, niggler, ot, prateekj, saurabh, shrikant, tambourine_man, thibaut_barrere, tjaerv, usaphp
  24 amerf1, coffeemug, dangoldin, neya, qiqing, signa11, twapi, w1ntermute, zrail
  23 Dekku, Shivetya, duggieawesome, ingve, jalanco, mikeevans, ninthfrank07, npalli, olalonde, playhard, rachbelaid, rb2e, subsystem
  22 adamnemecek, amarsahinovic, aritraghosh007, chmars, colinprince, ctoth, dutchbrit, eranation, jashkenas, jmacd, joshfraser, kine, lowglow, mike_esspe, muratmutlu, napolux, redDragon, rrhoover, tareqak, youngerdryas, yread, ytNumbers
  21 T-A, akandiah, anon1385, breck, dpaluy, glazskunrukitis, gridscomputing, homakov, ianstormtaylor, jusben1369, kunai, mindcrime, msvan, onosendai, peterkchen, prajjwal, pron, schrofer, sp332, treskot, ttunguz, urlwolf, ximeng
  20 BerislavLopac, BruceM, Jaigus, a3voices, abdophoto, angersock, boh, brandnewlow, chinmoy, chrisacky, cperciva, fekberg, ivoflipse, jasonshen, julien, llamataboot, mjn, mmahemoff, platz, shakes, smit, soundsop, speeder, tmoretti, tocomment, vellum
  19 RobAley, SanderMak, WadeF, bencevans, bitcartel, clarkm, declan, experiment0, gasull, goronbjorn, hartleybrody, jejune06, jkuria, k-mcgrady, kevingibbon, mijustin, minimaxir, njoglekar, obilgic, ohadfrankfurt, pclark, rainmaker23, robdoherty2, rubikscube, sciwiz, stevewilhelm, tdrnd, timf, twakefield, wamatt
  18 10char, ForFreedom, Pasanpr, aynlaplant, bsg75, cs702, deviceguru, dotmanish, ezl, jheitzeb, jstreebin, julien421, orrsella, pain_perdu, philfreo, polskibus, rblion, ridruejo, rmason, sew, swombat
  17 MaysonL, ananddass, barmstrong, bconway, bergie, crabasa, dmmalam, dsego, dshankar, equilibrium, fejr, ferdo, gebe, jkopelman, josephby, mikeleeorg, mkrecny, nashequilibrium, petenixey, petercooper, petrel, profquail, ra, replicatorblog, songzme, spking, taytus, ujeezy, wrongc0ntinent
  16 Adrock, Impossible, LiveTheDream, StavrosK, aashaykumar92, afshinmeh, ajaymehta, cdl, choult, damian2000, danboarder, daw___, electic, esolyt, followmylee, franze, giis, greenyoda, gregpurtell, hunvreus, jdorfman, jdp23, jellyksong, jorde, jordn, karamazov, linux_devil, lleims, mahmoudimus, maskofsanity, mjhea0, narad, nate, p4bl0, peter123, rfreytag, sciurus, sebg, sharkweek, spindritf, stevewillensky, tosh, tshtf, xmpir, zengr
  15 Ataub24, FredericJ, MIT_Hacker, Mithrandir, adulau, apress, areski, austenallred, babawere, bhauer, bitops, bmmayer1, bradleybuda, ckelly, cooldeal, czr80, dan1234, danielpal, gits1225, grey-area, groundCode, hamidr, hawkharris, jeffreyfox, lucb1e, mayop100, mikeknoop, pdknsk, relation, robbiet480, rodriguezcommaj, rpsubhub, sasvari, sheri, smacktoward, thejteam, thinkcomp, vinhnx, xtraclass
  14 0cool, Alex3917, ComputerGuru, Peroni, SparksZilla, amazedsaint, anons2011, apoorvamehta, bifrost, blacktulip, bradly, brandonb, bslatkin, c-oreills, carlosgg, charlieirish, chwolfe, codelion, davidroberts, dbaupp, dendory, dennybritz, draegtun, dsl, e1ven, edwintorok, enneff, glazemaster, grej, harryzhang, ishener, ivankirigin, japhyr, jarederondu, jkaljundi, johndcook, joshuacc, justincormack, kamaal, kanamekun, lisper, mathattack, networked, nickmain, pjvds, pragmatictester, pykello, robg, sabalaba, samspenc, sinnerswing, technologizer, ternaryoperator, thewarrior, timr, titlex, watermel0n, wlll
  13 6ren, AlexMuir, CrunchyJams, DavidChouinard, Kopion, acav, alter8, asanwal, benjlang, bra-ket, brettcvz, bryanh, bussetta, chrismealy, cinquemb, cpleppert, creamyhorror, cwilson, davidw, detcader, dfc, digisth, ekianjo, endtwist, gatsby, glaugh, gulbrandr, ics, jfaucett, jlongster, kn0thing, larrys, lenkendall, libovness, lukaseder, mactitan, marban, mgunes, michaelrbock, neeee, noinput, palidanx, paulschlacter, plessthanpt05, porker, pytrin, rjsamson, rohin, rpledge, samsolomon, sebkomianos, selmnoo, sethev, smaili, srathi, t0dd, techaddict009, tellarin, tilt, tlrobinson, tomasien, trendspotter, turoczy, vyrotek, waffle_ss, waterlesscloud, weu
  12 31reasons, ChuckMcM, CrazedGeek, DaNmarner, Ecio78, Fletch137, GuiA, ISL, IgorPartola, JacksonGariety, Kilo-byte, Rickasaurus, RockyMcNuts, Sealy, SteliE, aaronpk, abraham, achalkley, akos, alexholehouse, ankitoberoi, anujkk, bberson, bcl, bjansn, bjonathan, brianchu, btilly, bwertz, casca, cgi_man, changdizzle, chaz, chriscampbell, cocoflunchy, cramforce, d4vlx, daigoba66, dandrewsen, dctoedt, deusclovis, dmoney67, doh, dwynings, eladgil, enmaku, evolve2k, feelthepain, flippyhead, ghshephard, googletron, gz5, hansy, hkmurakami, hornbaker, irollboozers, jnoller, joxie, juliangamble, kareemm, kitcar, metajack, microwise, mindstab, mmariani, morisy, mpweiher, negrit, nekojima, nqureshi, paraschopra, pascal07, pauljonas, pbiggar, pearjuice, peterkelly, poissonpie, robheaton, seminatore, sherm8n, spooneybarger, stesch, taigeair, thegarside, theoutlander, tikhonj, tnorthcutt, uvdiv, vog, vu0tran, weisser, yeleti, zan2434
  11 Amadou, BIackSwan, Cieplak, Swizec, TDL, adrianhoward, aelaguiz, afschwartz, alok-g, aram, benhowdle89, benwerd, cstross, darxius, davidsmith8900, dboles99, decklin, devx, diggan, dkasper, dohertyjf, eduardordm, edw519, eibrahim, ekpyrotic, elleferrer, epenn, fatiherikli, frankdenbow, friism, guiseppecalzone, haven, hudibras, iand, igrigorik, immad, infoman, jacoblyles, jamest, jbaudanza, jdmitch, jgv, jipumarino, johnjlocke, jordanmessina, jpmc, judegomila, kloncks, knes, kogir, ldayley, leephillips, lest, lettergram, luigi, marcieoum, mazsa, memoryfailure, moonboots, morefranco, nbashaw, ndesaulniers, nonrecursive, nsns, nsp, null_ptr, octopus, oleganza, patrickaljord, pavel_lishin, philip1209, rasengan, rdemmer, remi, rfnslyr, rvivek, salimmadjd, sgrove, shared4you, sillysaurus2, simonreed, spdy, speednoise, stollercyrus, swannodette, teamgb, thehodge, tippytop, tlongren, tomse, tortilla, trevin, twog, ukd1, whit537, willvarfar, wiradikusuma, xijuan, yitchelle, zachinglis, zhs
  10 AirbnbNerds, Avalaxy, CaptainZapp, Danieru, JanLaussmann, MattRogish, Maven911, Mitt, SonicSoul, Xcelerate, andreiursan, andrewnez, antman, austinhallock, bdehaaff, bearwithclaws, bevenky, binarybits, blackhole, brown9-2, ceeK, chaostheory, cheeaun, chewxy, chrislloyd, clarky07, cobrausn, cyang08, dazbradbury, dcope, denysonique, dhotson, diminium, dirkk0, dmitri1981, dsowers, dylangs1030, dz0ny, edtechdev, fecak, felipebueno, fmavituna, giorgiofontana, gkuan, gregman, hboon, hippo33, hodgesmr, imkevinxu, jacobwg, jashmenn, jayadevan, jazzychad, jballanc, jf, jkarneges, josh2600, jpatokal, jvns, jwallaceparker, k33l0r, kafkaesque, kinlan, kintamanimatt, krat0sprakhar, kyledrake, lukedeering, maccman, malloc47, marojejian, matt1, maudlinmau5, mdturnerphys, mecredis, milesf, misiti3780, mjfern, molecule, mparramon, mschonfeld, nathanbarry, nileshd, ovechtrick, pavs, pearkes, petethomas, phenylene, philk10, pkrein, r4vik, radley, ramisms, robhawkes, rocky1138, rsobers, sidcool, siong1987, smanuel, stefan_kendall, sw007, swah, swampthing, swanson, sweis, techdog, thingsilearned, timjahn, tylermenezes, ukdm, will_brown, wsieroci, yesplorer, yonasb, zacharyvoase


Can you post a windowed version over say the last 3 months? Some of those accounts are very old.


I'm away from my computer right now, but I'll post something like that in a day or two.


does this project have a page?


I accidentally posted that with my other account. Yes - http://hackernewsletter.com


thx. is there an article on the aforementioned data?

(oh, and I submitted a link to your newsletter. maybe we can get it on the front page!)


Look through the submission list of each user whose posts make it to the top, or of all of the users submitting to new. You'll see if the stories each user posts gains lots of points; if they don't, it's all comment karma.


Any Karma measure analysis would be biased since Karma is accumulated through both Links and Comments (in contrast with Reddit, which splits both measurements).

"Dominated" is a flawed qualifier since Karma is not a zero-sum game.


There's a book you should read called "The 80/20 Principle". It might help you understand better how this sort of thing naturally happens, and why it is to be expected and nothing wrong about it.


No, they are dominated by the common denonimator and that is a massive flaw of vote-based user input. You will only see politically correct, left winged and in-tone liberal opinions and support everywhere. Mutual backpatting is encouraged and because of these traits, the circle continues.


echo chamber depending on first (maybe most numerous) adopters that self-reinforces by driving away dissenting voices. I agree 100%




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: