{"id":37,"date":"2026-11-02T18:14:04","date_gmt":"2026-11-02T18:14:04","guid":{"rendered":"https:\/\/classprojects.digitalwcu.org\/DESocial\/methods\/"},"modified":"2026-11-02T19:11:31","modified_gmt":"2026-11-02T19:11:31","slug":"methods","status":"publish","type":"page","link":"https:\/\/classprojects.digitalwcu.org\/DESocial\/methods\/","title":{"rendered":"Methods"},"content":{"rendered":"<p class=\"dh-eyebrow wp-block-paragraph\">DHM280 \u00b7 Project 2<\/p><p class=\"dh-lead wp-block-paragraph\">How a live stream of tweets became a downloadable monthly dataset, and what a researcher does with it afterwards.<\/p><figure class=\"wp-block-image size-full dh-hero dh-shot\"><img loading=\"lazy\" decoding=\"async\" width=\"1600\" height=\"700\" src=\"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-content\/uploads\/2026\/11\/site-methods.jpg\" alt=\"Screenshot of monthly COVID-19 and vaccine tweet ID dataset cards\" class=\"wp-image-55\" srcset=\"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-content\/uploads\/2026\/11\/site-methods.jpg 1600w, https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-content\/uploads\/2026\/11\/site-methods-300x131.jpg 300w, https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-content\/uploads\/2026\/11\/site-methods-1024x448.jpg 1024w, https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-content\/uploads\/2026\/11\/site-methods-768x336.jpg 768w, https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-content\/uploads\/2026\/11\/site-methods-1536x672.jpg 1536w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\" \/><figcaption class=\"wp-element-caption\">Each month has a full dataset and a vaccine-only dataset. Screenshot, Social Media Lab<\/figcaption><\/figure><div class=\"wp-block-group dh-section is-layout-flow wp-block-group-is-layout-flow\"><h2 class=\"wp-block-heading\">From stream to dataset<\/h2><ol class=\"wp-block-list\"><li><strong>Connect<\/strong> to Twitter&#8217;s COVID-19 Streaming Endpoint, which sends every public tweet Twitter tags as being about COVID-19.<\/li><li><strong>Collect<\/strong> around the clock with the lab&#8217;s script. The dashboard even tracks how many tweets came in each hour, which shows any gaps.<\/li><li><strong>Strip each tweet down<\/strong> to its ID, as Twitter&#8217;s rules require.<\/li><li><strong>Filter<\/strong> out a second, vaccine-only file using keywords.<\/li><li><strong>Summarize<\/strong> each month on the dashboard.<\/li><li><strong>Publish<\/strong> both files at the start of the next month.<\/li><li><strong>Researchers take over:<\/strong> they sample the data with the toolkit, then rehydrate it to get full tweets.<\/li><\/ol><\/div><div class=\"wp-block-group dh-section is-layout-flow wp-block-group-is-layout-flow\"><h2 class=\"wp-block-heading\">The critical step: the vaccine filter<\/h2><p class=\"wp-block-paragraph\">Most of this process just passes along what Twitter sends. Step 4 is different, because there the lab makes its own call about what counts as a &#8220;vaccine tweet.&#8221; A tweet goes into the vaccine file if it has a word starting with <strong>vaccin*<\/strong>, <strong>vacin*<\/strong>, or <strong>vax*<\/strong>. The asterisk catches any ending, so it picks up vaccine, vaccinated, vax, vaxxed, and so on. Including &#8220;vacin*&#8221; also catches a common misspelling, which is a thoughtful touch.<\/p><p class=\"wp-block-paragraph\">This matters because vaccine misinformation was some of the most harmful content of the pandemic. A ready-made vaccine file lets a communication researcher go straight to that conversation instead of filtering 100 million tweets themselves.<\/p><p class=\"wp-block-paragraph\">Keyword filters are blunt, though. A tweet about getting &#8220;the jab&#8221; or &#8220;my Pfizer shot&#8221; gets missed, and so do tweets written in other languages. Meanwhile, a tweet that mentions vaccines in passing gets pulled in. Anyone using the file needs to know it is a starting point, not a perfect set.<\/p><p class=\"wp-block-paragraph\">This step happens inside the collection script described on <a href=\"\/DESocial\/tools\/\">Tools<\/a>, and it produces the second monthly file described on <a href=\"\/DESocial\/data\/\">Data<\/a>.<\/p><\/div><div class=\"wp-block-group dh-section is-layout-flow wp-block-group-is-layout-flow\"><h2 class=\"wp-block-heading\">How good is the documentation?<\/h2><p class=\"wp-block-paragraph\">For the step the lab controls, it is excellent. The exact keyword rule is written out on the <a href=\"https:\/\/covid19misinfo-tweets.socialmedialab.ca\/tweet_ids\" target=\"_blank\" rel=\"noopener\">Downloads page<\/a>, so anyone can understand the vaccine file or repeat the filter on their own data.<\/p><p class=\"wp-block-paragraph\">Steps 1 and 2 are a different story. They rely on Twitter&#8217;s internal rules for what counts as a COVID tweet. The lab points to Twitter&#8217;s documentation, but that endpoint is gone now, so this part of the method is a black box the lab never controlled.<\/p><\/div><p class=\"dh-pager wp-block-paragraph\"><a href=\"\/DESocial\/tools\/\">\u2190 Tools<\/a><a href=\"\/DESocial\/research-values\/\">Research Values \u2192<\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>DHM280 \u00b7 Project 2 How a live stream of tweets became a downloadable monthly dataset, and what a researcher does with it afterwards. From stream to dataset The critical step: the vaccine filter Most of this process just passes along what Twitter sends. Step 4 is different, because there the lab makes its own call [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":7,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-37","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/pages\/37","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/comments?post=37"}],"version-history":[{"count":2,"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/pages\/37\/revisions"}],"predecessor-version":[{"id":86,"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/pages\/37\/revisions\/86"}],"wp:attachment":[{"href":"https:\/\/classprojects.digitalwcu.org\/DESocial\/wp-json\/wp\/v2\/media?parent=37"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}