The Zipf Mystery

Vsauce

19 M megtekintés1 696

    The of and to. A in is I. That it, for you, was with on. As have ... but be they.
    RELATED LINKS AND SOURCES BELOW!
    tweetsauce
    electricpants
    WordCount.org www.wordcount.org/
    How many days have you been alive? www.beatcanvas.com/daysalive.asp
    random letter generator: www.dave-reed.com/Nifty/randSeq.html
    Dictionary of Obscure Sorrows: hufast.info
    Word frequency resources:
    [lemmatized] en.wikipedia.org/wiki/Most_common_words_in_English
    www.uow.edu.au/~dlee/corpora.htm
    www.wordfrequency.info
    www.anc.org/data/anc-second-release/frequency-data/
    www.titania.bham.ac.uk/docs/
    www.kilgarriff.co.uk/bnc-readme.html#raw
    en.wiktionary.org/wiki/Wiktionary:Frequency_lists
    ucrel.lancs.ac.uk/bncfreq/
    [PDF] www.wordfrequency.info/files/entries.pdf
    [combined Wikipedia and Gutenberg] www.monlp.com/2012/04/16/calculating-word-and-n-gram-statistics-from-a-wikipedia-corpora/
    corpus.byu.edu/coca/files/100k_samples.txt
    corpus.byu.edu/
    corpus.leeds.ac.uk/list.html
    books.google.co.uk/books?id=ja1_AAAAQBAJ&dq=word+frequency+coca&lr=
    www.ling.helsinki.fi/kit/2009s/clt231/NLTK/book/ch01-LanguageProcessingAndPython.html
    Great Zipf's law papers:
    colala.bcs.rochester.edu/papers/piantadosi2014zipfs.pdf
    www.ling.upenn.edu/~ycharles/sign708.pdf
    arxiv.org/pdf/cond-mat/0412004.pdf
    www-personal.umich.edu/~mejn/courses/2006/cmplxsys899/powerlaws.pdf
    Zipf’s law articles and discussions:
    www.theatlantic.com/magazine/archive/2002/04/seeing-around-corners/302471/
    io9.com/the-mysterious-law-that-governs-the-size-of-your-city-1479244159?.Ej9760cOTJCPS_Bq4mjoww.0
    plus.maths.org/content/os/latestnews/may-aug08/food/index
    judson.blogs.nytimes.com/2009/05/19/math-and-the-city/?em
    plus.maths.org/content/mystery-zipf?src=aop
    www.datasciencecentral.com/profiles/blogs/why-zipf-s-law-explains-so-many-big-data-and-physics-phenomenons
    en.wikipedia.org/wiki/Zipf%27s_law
    books.google.co.uk/books?id=f8GrzlnMSm8C&pg=PA62&redir_esc=y#v=onepage&q&f=false
    arxiv.org/pdf/0802.4393v1.pdf
    www.pnas.org/content/108/9/3526.full
    lewisdartnell.com/language_page.htm
    wugology.com/zipfs-law/
    other Zipf’s law PDFs
    ftp.iza.org/dp3928.pdf
    arxiv.org/pdf/1402.2965.pdf
    arxiv.org/pdf/1104.3199.pdf
    www.lel.ed.ac.uk/~jim/zipfjrh.pdf
    www.ncbi.nlm.nih.gov/pmc/articles/PMC2834740/#pone.0009411-Mandelbrot1
    polymer.bu.edu/hes/articles/pgs02a.pdf
    in untranslated language: arxiv.org/pdf/0808.2904.pdf
    pages.stern.nyu.edu/~xgabaix/papers/zipf.pdf
    www.hpl.hp.com/research/idl/papers/ranking/ranking.html
    statweb.stanford.edu/~owen/courses/306a/ZipfAndGutenberg.pdf
    arxiv.org/pdf/1310.0448v3.pdf
    www.kornai.com/Papers/glotto5.pdf
    Zipf’s law slides:
    www.slideshare.net/guest9fc47a/nlp-new-words
    Pareto Principle and related ‘laws’:
    www.squawkpoint.com/2013/03/pareto-principle/
    billyshall.com/blog/post/paretos-principle
    en.wikipedia.org/wiki/Pareto_principle
    Random typing and Zipf:
    www.longtail.com/the_long_tail/2006/09/is_zipfs_law_ju.html
    health 80/20: archive.ahrq.gov/research/findings/factsheets/costs/expriach/expriach1.html
    Principle of least effort:
    en.wikipedia.org/wiki/Principle_of_least_effort
    en.wikipedia.org/wiki/Satisficing
    www.pnas.org/content/100/3/788.full.pdf [PDF]
    csiss.org/classics/content/99
    self organized criticality:
    journal.frontiersin.org/article/10.3389/fnsys.2014.00166/full
    Hapax Legomenon:
    campus.albion.edu/english/2011/02/15/hapax-legomenon/
    www.dailywritingtips.com/is-that-a-hapax-legomenon/
    en.wikipedia.org/wiki/Hapax_legomenon
    [PDF] www.aclweb.org/anthology/J10-4003
    www.wired.com/2012/01/hapax-legomena-and-zipfs-law/
    oed.hertford.ox.ac.uk/main/content/view/402/450/index.html#_ftn1
    oed.hertford.ox.ac.uk/main/content/view/36/166/index.html
    Learning curve: en.wikipedia.org/wiki/Learning_curve
    Forgetting curve:
    www.trainingindustry.com/wiki/entries/forgetting-curve.aspx
    en.wikipedia.org/wiki/Forgetting_curve
    Experience curve effects: en.wikipedia.org/wiki/Experience_curve_effects
    Forgetting
    and zipf's law: act-r.psy.cmu.edu/wordpress/wp-content/uploads/2012/12/37JRA_LS_PS_1991.pdf
    public.psych.iastate.edu/shacarp/Wixted_Carpenter_2007.pdf
    marshalljonesjr.com/youll-remember-less-than-001-of-your-life/
    en.wikipedia.org/wiki/Forgetting
    www.reddit.com/r/Showerthoughts/comments/3gu9qk/it_only_takes_three_generations_for_you_to_be/
    music from:
    hufast.info
    www.audionetwork.com

    Közzététel: 5 évvel

    Megjegyzés

    1. Hayato Tenryuu

      kdasj fkhdsi; kdnwejf; saijf sdmfjsda; fjkl huijosd hfjmsbfjksl i zfkjmje snj fhgeuj fusiod fkjmsd hjfsd fjksdl; fjks bfnjs bfvgudsxfv usdfzjk dns mklf hesdujvfifsd zkjfl;dhsjk mZfbjsl fujsd fjsdm hzfjdlhv iujsd fjmsd ;b jffls duvjsd jfghsdjv l sjkfhsd j FBvhdjsl VHfhdsLK F This can't be Zipfy. After all, I have only 2 thumbs to hit the space bar with, and 8 fingers to hit other letters with. So I am actually more likely to have short words, but VERY short words are rather unlikely due to the time it would take to use a thumb twice rather than use any of the other four fingers instead of the second thumb stroke. Also 16:15 I'm not sure that note is legal. If we are talking about words, I would expect to talk about _each unique word manifestation_ . Otherwise you are just defining the rank of certain types of words, like "article adjectives are the most common, prepositions are the second most, conjuntions are the third most," rather than the words themselves. That is just an illustration of how human communication trends are, which is obviously independent of language itself, because humans all conceptualize and use language the same way. "Say the most content, with a limited density, a maximum simplicity, and the shortest amount of time/syllables possible". And that's just Dijkstra's algorithm, with a few modifications I'm sure. Quizzaciously is just a hard word to say, and it's long. So, of course we wouldn't use it. If anything, Zipf-ian systems are just systems of priority, like Dijkstra's algorithm. You pick the favorable quality, and you ignore the rest because you don't have the processing power to keep it all. You forget the mundane, but all things deemed relevant or important are remembered. And occasionally, random facts are also randomly selected to be "included". If I'm a hiring manager, and I want to hire someone with X, Y, and Z qualities, then whatever resumes I see with X, Y, and Z will get the most of my time, and those without one or more of those desirable traits will receive very little, if any, of my time. It's just a prioritization system. A naturally occurring prioritization system.

    2. Andrew Qi

      80% of the battery usage is by 20% of the processes.

    3. Ikenna Okonkwo

      sauce

    4. Dead Guy Dude Man

      Was "the" a pun?

    5. Jacob Rogers

      I wonder what a book that was intentionally written to not follow this would be like?

    6. Jonah Falcon

      Wordcount uses Flash. And they haven't updated the site.

    7. Aladen Zannerni

      Isn't this mathematically normal? the total odds/frequency of anything will be 1 and when ranked by order in a graph will appear the same way? otherwise the total odds will either exceed or never reach 1, which can't happen.

    8. Prakhar Sharma

      The most thought word must be "F" word though,🤔

    9. Bodor János

      I think the most used Hungarian word is "B@zdmeg" (i needed to censor it). It's a swear word. And it's beautiful.

    10. Dron008

      "Quizzaciously" is more popular now. There is an article in Urban Dictionary.

    11. Somenath Garai

      duh!?

    12. kiro cazam

      wait...so does this mean 20% of all youtubers are actually making videos and are popular while 80% are only watchers ? does this means 20% out of those who make videos in youtube are popular and 80% are barely known ? does this mean 20% out of all of the comments in a video gets popular while 80% just fly by and barely get over 5 likes ? ...does this mean 20% out of everyone in this world...get rich while 80% just die poor does this mean 20% of people in this world die alone while 80% the opposite ? 20 80 rule is...scary and weird tbh it's just way too weird to think about it...this thing really did caught my attention

    13. aabic

      Any German here? I Like the Title😂

    14. Anex Antony

      Ancient language that hasn't been translated yet-

    15. 5MinPhil

      Wth is this wizardry! I was just thinking this morning about this vsauce video I’d seen years ago but I couldn’t remember what it was called! I only remembered the final quote from Emerson. And HUfast just suggest this to me today. 🤯

    16. Doby Adoby

      Vsauce: sauce is the 5,555 th mosts common english word Also Vsauce every video: 0:00

    17. Undefind

      The 45° angle line we get when graphing the frequency of words is not just for humans. The different sounds made by Dolfin show this 45° angle pattern as well.

    18. Dish Soap Gaming

      80% of my brain is fried by the first 20% of this video

    19. N!cky

      Idk if you're reading comments but what do you think, would a auto suggest feature on keyboard follow zipfs law. If we humans follow his law everywhere what's the possibility of computer with auto suggest for words doing the same? (i mean there is chance it would because we created it)

    20. Zach Schmied

      So, flipping someone the bird while driving is responsible for about 95% of the road rage that occurs. Makes sense...

    21. Robcop Westton

      Most things go Top to bottom, you don't start a cake with a "hint of cinnamon " or " a cherry" even someone doing there accounts would start with big number first.

    22. Pigbelly19 ‘

      would have been funny if he didnt say the for the entire video

    23. Jacob Costen

      mfw i suddenly realise the background music at 12:45 is actually bach double but jaunty piano mode

    24. ِ

      Yeah its simple.

    25. Toms Markovs

      Where did I put my paper clips?

    26. ِ

      Answer next tk questions

    27. Vineet Sharma

      A similar law is Benford's Law.

    28. Henry Anderson

      me and the bois going on to wikapedia and making scauce the 69 most used word

    29. TheMockingJay

      Reading up on superspreading events and epidemiology, the 80/20 rule was also observed -- 80% of disease transmission is caused by 20% of infected cases

    30. Cameron Goodall

      there is no way "school" is uses more than "no"

    31. blocboy jb

      The Bible was written by man bc it follows ziffs law

    32. Matt Garrelts

      we bouta get quizzacious

    33. Andrej Ajnik

      This is Benford's law

    34. Andrew Zeitler

      The absent reason putatively guide because transaction regretfully milk besides a overrated territory. aboard, tranquil skirt

    35. Dennis Garage

      Sounds very similar to Benford's law also. There seems to be a more primary factor causing the occurrence of this throughout various systems. Living in a simulation becomes more plausible the more you realize the patterns.

    36. Lu Bu

      I think this all comes down to usefulness and meaning. The could be used to pinpoint any other word while sauce is so specific that it isn’t used as much because the explanation is direct and intentional. The is what exactly? but sauce is sauce.

    37. ABC DEF

      80% of my stories are filled with um and 20% of my stories are actually told

    38. John Heggerty

      Dude, after 2 video spirals on your channel I'm not ashamed to admit that although chunks of the content my idiot brain has no way to comprehend it. But the way you present and explain the info, what I end up being able to fathom, blows my mind! Love your content.

    39. Shard FloofFox

      Bruh

    40. Drooby Dooby Doo

      We live in a simulation.

    41. Tuckster04

      Who else tried to count the amount of times he said "the"

    42. Edward Trevino

      This 80/20 split forecasted Tfues contract

    43. Aljulanda Almaghadari

      It indicates that there is one creator of the universe.. The One and Only God

    44. cant run

      All I see is FRACTALS.

    45. Kjonesey ‘

      3:03 wow

    46. Elise Visser

      80 percent of my test is about 20 percent of what we had to study

    47. a a

      Meanwhile: German *der die das die den die das die dem der dem den des der des der*

    48. Ryan Cooper

      since this cideo was published, Quizzaciously has gone from 1 result, to 17,500 results. Proving your point that the more something is used, the more it be used again.

    49. Nooby-kun

      2:08 normal people: wow people really like sauce. me being a weeb: ( ͡° ͜ʖ ͡°)

    50. Cedrick Narsiso

      Zipf's Law is just the algorithm that God put into a lot of shit after a long day of developing the " everything"

    51. toastboii

      Vsauce always makes me feel philosophical

    52. E

      my grandma did a funny dance on the floor and i dont know why vsauce help

    53. papinbala

      in the beginning was the word, and the word was with God and the word was God

    54. James Taylor

      Is this proof that we’re in a simulation?

    55. pizzasmithdude

      This law explains why you can never solve the problem of inequality in every single measurement of socio-economic status.

    56. Mark Prins

      I think that Zipf was correct. Speaking more than ten languages myself, I've come to the theory that language is just a way to transfer thoughts. In doing so you will need a (rare) specific word and several (common) words to describe how people should interpret the specific word.

    57. Lady Ymir

      "[...] in *the* english language. About one in every 16 words you encounter on a daily basis is *the* ." son of a...

    58. Deconstruct//Recreate

      Hapax Legomenon derives from the Greek Άπαξ λεγόμενον which literally means “ said once”.

    59. Brave Cat

      so then what happens when 99% of the wealth is with 1% of the people. such a huge imbalance im sure Cthulu will come and have to balance that out. or maybe thats why the sun is freaking the fuck out and is about to spit a micro nova at us.

    60. KUWALDA

      michael saying "80 percent and..." so much fucked me over ngl

    61. Akyllia

      It is funny how the Pareto principle is relevant today. Here in Quebec, Canada, they said that 20% of the people at risk of contracting the COVID-19 virus (mostly people over 65 years old) represent 80% of the people hospitalized in our hospitals! I didn't know about that principle then!

    62. Qwerty

      too bad i can't check out wordcount since flash player is dead

    63. Mr. E

      Kinda puts the kibosh on the whole conservative assertion that the wealthy have what they have because they "deserve it". Seems more likely that they're just lucky beneficiaries of some cosmological constant that had little to nothing to do with their effort. This of course begs the question: if you're no more entitled to wealth or power than the next person, *why do we constantly accept systemic wealth and power inequality?*

    64. Will It Snap?

      "Zipf-y"

    65. Junovah Hunziil

      Vsauce Say "I want" and then everything after it without (The)

    66. Complex God

      What a video!!!

    67. Scott Keselowski

      Video must be old. The most common word now is literally :p

    68. fewbronze games

      everyone use the second most used word as much as possible

    69. Itchy Eyes McRealize

      This can't be right ....

    70. Joe Blackmore

      I know use quizatiously once at least a day I no speel

    71. The Failed_safe

      Well order is the last thing that you'd think chaos would do, so thats exactly what it did

    72. Hugo Le Roy

      Have you forgotten this video now that 5 years have passed?

    73. SoftAlexandra3 Plays

      He got us in the first half, not gonna lie.

    74. Helpful Sysops

      0:26 This is a really very good Poem.

    75. Robin

      "The" dont exist in my language. You change the word instead that wold be after "the".

      1. Sander Bouwhuis

        1. Interesting, which language is that? 2. Languages don't necessarily agree on the order. Inuits probably have many words that are frequently used like snow/white/cold, while the Bedouin probably have very few words like that. On the other hand, Bedouins probably have and use many words related to yellow/sand/hot, while the Inuits probably use these rarely if at all.

    76. Ethereal Sky Daddy

      This is like the Fibbonaci Sequence.

    77. Sebastian Solis

      What if we tend to use more words than others because some might feel repetitive? If someone puts "the" 30 times in an essay it will certainly look better than if they put "furthermore" also 30 times in it.

    78. tdchewy

      Your science is flawed as only a few letters on their own are actually words. F or Q are not words for example, they are letters.

    79. [GD] Goose

      After the intro his percentage when the up to 7%

    80. Cattu Doggu

      Im scared of this \ line now

    81. Mag Magnet

      14:08 imagine this in a bernie sanders voice LOL

    82. Jayleigh

      Okay but Vilfredo Pareto has to be the coolest name ever

    83. Double A-AA

      every time micheal says the word "the" I go: "Micheal stop! STOP! You're killing me!"

    84. Eric Young

      me: how is this possible? Vsause: yes

    85. Benjamin Coffman

      V-sauce. Really? "Its like if you take a snowball and roll it down a snowy hill." Yes my genius friend, the snowball effect.

      1. Benjamin Coffman

        Still love you Michael :p

    86. Liam Binsfeld

      Haha the

    87. Eye Master 3.14

      So, quizzaciously is a fun word, but can it beat fabaceous (Adv. of the nature of the bean)?

    88. InsteadImAwake

      Quizzaciously

    89. qwzerty_

      That intro is what makes time continue

    90. Bernice Gardner

      The better quart enzymatically whistle because flugelhorn likely shade under a onerous care. boring, mushy link

    91. Ultimate Flan

      11:27 top ten anime plot twists

    92. afeelfortheflow with Manic

      This guy is extraordinary

    93. Magic Beans

      AAAAAAAA! Vsauce is making me interested in math! NOOOOO What do I do lol

    94. Cielo

      so if i study onlky 20% of most used words & grammar i can learn 80% of a language 🤑

    95. A1 Helix

      This is like almost actual proof we are in a simulation

    96. Vivek Saraswat

      Since there are an infinite causes and effect pairs, I believe every arbitrary combination like 30/70 or 50/50 will also have infinite occurences (there may not be anything special about 20/80). If someone shows that the popularity or frequency of a particular combination, say 20/80, is more compared to the rest of the possibilities, then it would be something weird. Listing a 100 examples which follows 20/80 principle from an infinite more total examples does not lead me to believe that this combination is special. Nice video though, as always 😉.

    97. Thomas Beasley

      We broke the law!!! Only 9 percent of plastic is recycled 91 percent is in landfills or is litter.

    98. {DavidDoesn'tKnow}

      80% of my life is boring but 20% is more boring.

    99. THE JACK HQ

      We have the Illuminati all figured out now 😏. They thought they could pull the Zipf over our eyes. Not so much anymore.

    100. Hugo Petrarolha

      4:23 George Zipf é a cara do jornalista Augusto Nunes.