Hacks, Leaks, and Revelations: The Art of Analyzing Hacked and Leaked Data

Data-science investigations have brought journalism into the 21st century, and—guided by The Intercept ’s infosec expert Micah Lee— this book is your blueprint for uncovering hidden secrets in hacked datasets. Unlock the internet’s treasure trove of public interest data with Hacks, Leaks, and Revelations by Micah Lee, an investigative reporter and security engineer. This hands-on guide blends real-world techniques for researching large datasets with lessons on coding, data authentication, and digital security. All of this is spiced up with gripping stories from the front lines of investigative journalism. Dive into exposed datasets from a wide array of sources: the FBI, the DHS, police intelligence agencies, extremist groups like the Oath Keepers, and even a Russian ransomware gang. Lee’s own in-depth case studies on disinformation-peddling pandemic profiteers and neo-Nazi chatrooms serve as blueprints for your research. Gain practical skills in searching massive troves of data for keywords like “antifa” and pinpointing documents with newsworthy revelations. Get a crash course in Python to automate the analysis of millions of files. You will also learn how to: Master encrypted messaging to safely communicate with whistleblowers. Secure datasets over encrypted channels using Signal, Tor Browser, OnionShare, and SecureDrop. Harvest data from the BlueLeaks collection of internal memos, financial records, and more from over 200 state, local, and federal agencies. Probe leaked email archives about offshore detention centers and the Heritage Foundation. Analyze metadata from videos of the January 6 attack on the US Capitol, sourced from the Parler social network. We live in an age where hacking and whistleblowing can unearth secrets that alter history. Hacks, Leaks, and Revelations is your toolkit for uncovering new stories and hidden truths. Crack open your laptop, plug in a hard drive, and get ready to change history. Data-science investigations have brought journalism into the 21st century, andguided by The Intercept s infosec expert Micah Lee this book is your blueprint for uncovering hidden secrets in hacked datasets. In the current age of hacking and whistleblowing, the internet contains massive troves of leaked information. These complex datasets can be goldmines of revelations in the public interest if you know how to access and analyze them. For investigative journalists, hacktivists, and amateur researchers alike, this book provides the technical expertise needed to find and transform unintelligible files into groundbreaking reports. Guided by renowned investigative journalist and infosec expert Micah Lee, who helped secure Edward Snowdens communications with the press, youll learn the tools, technologies, and programming basics needed to crack open and interrogate datasets freely available on the internet or your own private datasets obtained directly from sources. Each chapter features hands-on exercises using real hacked data from governments, companies, and political groups, as well as interesting nuggets from datasets that never made it into published stories. Youll dig into hacked files from the BlueLeaks law enforcement records, analyze social-media traffic related to the 2021 attack on the U.S. Capitol, and get the exclusive story of privately leaked data from anti-vaccine group Americas Frontline Doctors. Along the way, youll Introduction Part 1: Sources and Datasets Chapter 1: Protecting Sources and Yourself Chapter 2: Acquiring Datasets Part 2: Tools of the Trade Chapter 3: The Command Line Interface Chapter 4: Exploring Datasets in the Terminal Chapter 5: Docker, Aleph, and Making Datasets Searchable Chapter 6: Reading Other People's Emails Part 3: Writing Code Chapter 7: An Introduction to Python Chapter 8: Working with Data in Python Part 4: Structured Data Chapter 9: BlueLeaks, Black Lives Matter, and the CSV File Format Chapter 10: BlueLeaks Explorer Chapter 11: Parler, the Insurrection of January 6, and the JSON File Format Chapter 12: Epik Fail, Extremism Research, and SQL Databases Part 5: Case Studies Chapter 13: Pandemic Profiteers and COVID-19 Disinformation Chapter 14: Neo-Nazis and Their Chat Rooms Afterword Appendixes Appendix A: Using the Windows Subsystem for Linux Appendix B: Scraping the Web Introduction Part I: Sources and Datasets Chapter 1: Protecting Sources and Yourself Chapter 2: Acquiring Datasets Part II: Tools of the Trade Chapter 3: The Command Line Interface Chapter 4: Exploring Datasets in the Terminal Chapter 5: Docker, Aleph, and Making Datasets Searchable Chapter 6: Reading Other People’s Email Part III: Python Programming Chapter 7: An Introduction to Python Chapter 8: Working with Data in Python Part IV: Structured Data Chapter 9: BlueLeaks, Black Lives Matter, and the CSV File Format Chapter 10: BlueLeaks Explorer Chapter 11: Parler, the January 6 Insurrection, and the JSON File Format Chapter 12: Epik Fail, Extremism Research, and SQL Databases Part V: Case Studies Chapter 13: Pandemic Profiteers and COVID-19 Disinformation Chapter 14: Neo-Nazis and Their Chat Rooms Afterword Appendix A: Solutions to Common WSL Problems Appendix B: Scraping the Web Index