Scraping law, for people who are not lawyers or software engineers
The LexLint Scraping Brief
There is no internet law. There is the law of every place your scraper touches, and this is how to find out which. Seven documents in three parts, the last a walkthrough that asks twelve questions and says what to worry about first, and a two-page sheet for your lawyer.
Start the walkthrough Read: there is no internet law
Legal information, not legal advice. These documents describe the law as published and dated. They do not apply it to any project; that is a question for a lawyer.
This finds the obvious problems. It does not find all of them, and it does not clear a project.
About this sectionUpdated ShowHide
Sean McDermott, Co-Founder and CEO, UnGovr
Written by Sean McDermott (with AI assistance) using the LexLint law library, which supplied every legal instrument, status and date on these pages, and the handbook and insight documents on lexlint.io that carry the depth behind each one.
Every law named here links to its summary page on lexlint.io, translated to English (if needed) and restructured to a standard format for human and code use. Every case links to the court's or the regulator's own record where one could be reached.
© 2026 UnGovr, publishing as LexLint. The text and the figures are licensed under Creative Commons Attribution-ShareAlike 4.0: share and adapt them, including commercially, with credit to LexLint (UnGovr) and under the same licence. Please contact LexLint at hello@ungovr.org to discuss other terms. Logos and wordmarks belong to their owners.
Corpus figures as of .
The brief, in reading order
- Part 1 Start here Whose law is this, anyway?
- Part 2 The law What can go wrong?
-
Part 3
Doing it
What do I do now?
Read beside it: the two-page scraping sheet and the glossary.
Every document draws on the LexLint software-law corpus and on the handbook and insight documents that carry the depth behind each one. Every law named links its own page, with its status and the date it was read.
Start here
Whose law is this, anyway?
-
Document 1
Is web scraping legal? · there is no internet law, only the law of every place your scraper touches
No body of law governs the internet. A scraper is under the law of every place it touches, and the places are found by asking where each party is: you, the machine, the site, the people on its pages, and the model if one is in the loop.
Read this if you are about to collect data from other people's websites and want to know which law applies before you start.
You come away with the parties around a scraper, the three questions that find the places, and what scraping shares with the law of AI agents and what is its own.
-
Document 2
Which country's law applies to my scraper? · three places, and how to find the third
Where you are, where the scraper runs, and where the target site has legal standing. The first two you know. The third takes a method, and the method is a ladder of evidence from a legal notice down to a domain ending.
Read this if you know which sites you want and need to say which places' law you are under, including for a site whose owner you have never looked up.
You come away with the three places, what the target site's place turns on for each body of law, the evidence ladder in the order to trust it, and what to do when you cannot tell.
The law
What can go wrong?
-
Document 3
Can I scrape public data? · public is not the same as permitted
A page anyone can see is a fact about how you got the data, not a permission to have it. Privacy law, copyright and computer-misuse law each answer the question differently, and only one of them is on your side.
Read this if you were told scraping is fine if the data is public and want to know how far that is true.
You come away with three bodies of law, three meanings of public, and the one where public helps you.
-
Document 4
Can I get past robots.txt, a CAPTCHA or a login? · what is in the way, and what it means to get past it
A robots.txt file asks. A rate limit pushes back. A bot challenge and a CAPTCHA stand in the way. A login and a terms page make a contract. A letter revokes. Each rung moves you into a different body of law, and a court lit one rung in July 2026.
Read this if you hit something in the way and want to know what stepping around it would mean.
You come away with the eight rungs from weakest to strongest, what each is in law, which body of law each moves you into, and where a careful crawler stops.
-
Document 5
Can I scrape names, emails and photos? · the people in the data
Consent is not available to a scraper. Legitimate interest is the basis with a test attached. The duty to tell people does not go away because you cannot tell them one by one. And “they put it online” is an argument about one limb of the test, not a way around it.
Read this if the pages you want have people on them: names, contact details, faces, posts.
You come away with why consent is out, what the legitimate-interest test asks, the notice you owe, where the regimes diverge, and the enforcement record.
Doing it
What do I do now?
Read beside it: the two-page scraping sheet and the glossary.
-
Document 6
How to scrape responsibly · what each rule buys you in law
Ask first. Identify yourself. Slow down. Keep a record. Do not take what is clearly not public, and do not pass on what you have no right to. Each is good manners, and each one closes a legal door.
Read this if you have decided to go ahead and want to do it in the way that keeps the obvious problems off the table.
You come away with seven rules, the legal reason for each, the record to keep, and what to take to a lawyer.
-
Document 7
What do I need to worry about? · twelve questions
Answer twelve questions about what you want to scrape, from where, and what you will do with it. The result is the short list of things to be concerned about, in the order to deal with them, and a profile you can hand to a developer or a lawyer.
Read this if you want the short list, not the reading.
You come away with the places, the concerns in order, and the profile.
Which document do I need?
| Your question | Start with |
|---|---|
| I want to know whether scraping a site is legal at all | 1, the parties |
| I want to work out which country's law applies | 2, the places |
| I want to scrape data that is public | 3, public data |
| I want to get past robots.txt, a block, a CAPTCHA or a login | 4, the barriers |
| I want to collect names, emails, photos or posts | 5, the people |
| I want to do it properly | 6, careful scraping |
| I want to get the short list and nothing else | 7, the walkthrough |
| I want to print two pages for my lawyer | the scraping sheet |