mapscout
Find every business of one kind in one area on Google Maps, and get them in an Excel workbook.
mapscout scrape "Thaltej, Ahmedabad" -c "software companies"For each business: name, categories, full address, coordinates, phone, website, rating and review count, opening hours, price range, attributes, booking links, owner status, a few reviews, and the emails and social profiles found on the business's own website.
Installation
Works on Windows, macOS and Linux. You need an internet connection and about 500 MB of disk space, mostly for the Chromium browser.
1. Install uv
uv installs the right Python version and every library for you.
Windows (PowerShell):
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"macOS or Linux:
curl -LsSf https://astral.sh/uv/install.sh | shClose and reopen the terminal afterwards so the uv command is found.
2. Get the code
git clone https://github.com/<your-username>/mapscout.git
cd mapscoutOr download the ZIP from GitHub, unzip it, and open a terminal in that folder.
3. Install the libraries and the browser
uv sync
uv run playwright install chromiumOn Linux, if Chromium reports missing system libraries, use uv run playwright install --with-deps chromium.
4. Make mapscout work in any terminal
uv tool install --editable .Open a new terminal and check:
mapscout --helpSkipping this step is fine: run every command from the project folder with uv run in front, e.g. uv run mapscout scrape.
Run the install commands in a normal terminal (PowerShell, Windows Terminal, macOS Terminal). Installs made from inside some desktop apps' sandboxed terminals can end up invisible to your other terminals.
Updating
git pull
uv syncCode changes take effect immediately with the editable install. Only if the dependencies changed, reinstall the command:
uv tool install --editable . --reinstallUninstalling
uv tool uninstall mapscoutThen delete the project folder. Your results stay in the data folder until you delete it too.
Your first run
- Type
mapscout scrape. It asks three questions:What should I search? Leave Business type empty for all 63 common types, or separate several with commas. Area e.g. Vesu: Thaltej City e.g. Surat: Ahmedabad Business type (optional) e.g. software companies, tech companies: software companies - It looks the area up on OpenStreetMap and shows the plan: the matched area, its size, and how many searches it will start with.
- Progress bars appear for Searches, Place pages and Websites. They all move at once.
- At the end you get a table of what's done, a run report, and the path of the Excel file.
Start small. Add --max-places 20 to your first run to see the output in a few minutes before committing to a long run.
Use Google's name for the category: "software companies", "dentists", "hair salons". Vague terms still work ("tech companies" is read as Software company).
Commands
| Command | What it does |
|---|---|
mapscout scrape | Start a new search. Asks for whatever you don't pass. |
mapscout resume <job> | Continue a stopped or blocked job. Failed items get another try. |
mapscout jobs | List all jobs and their progress. |
mapscout report <job> | What a job did and where the time went. |
mapscout export <job> | Write the Excel file again. |
mapscout reparse <job> | Re-read saved page data after a parser fix, without searching again. |
mapscout grid "<area>" | Preview the search grid for an area. Searches nothing. |
mapscout check | Health check: do Google's pages still come through correctly? |
Every command has --help, e.g. mapscout scrape --help.
mapscout scrape
Searches an area for one or more kinds of business and writes everything to Excel.
# Ask for area, city and business type
mapscout scrape
# Everything up front, no questions
mapscout scrape "Thaltej, Ahmedabad" -c "software companies"
# Several kinds of business
mapscout scrape "Koregaon Park, Pune" -c "cafes,restaurants,gyms"
# Only ask for the area and city
mapscout scrape -c dentistsThe area is written "<area>, <city>". Add the state if the name is ambiguous: "Vesu, Surat, Gujarat".
Each run gets a job number, printed at the start; the other commands use it.
mapscout resume
mapscout resume 12Picks up exactly where job 12 stopped: after Ctrl+C, a crash, a lost connection or a Google block. Anything that failed before gets a fresh set of tries. The job keeps the options it was started with.
mapscout jobs and mapscout report
mapscout jobs
mapscout report 12jobs lists every job with its area, number of business types, status (running, done, interrupted, blocked), businesses done and start time.
report shows the job's counts per stage and its run report: time spent, searches per minute, searches stopped early,
businesses skipped as another kind (with their categories), read / reused / taken from search results, how many pages got Google's full view,
failures, automatic slow-downs and blocks.
mapscout export and mapscout reparse
mapscout export 12
mapscout export 12 --out "D:\reports\thaltej.xlsx"
mapscout reparse 12export rewrites the workbook from the database, e.g. after you closed a file Excel had locked, or into another location.
reparse re-reads the raw page data saved during the run. Use it after fixing parsing.py when Google changed its data layout, then run export.
mapscout grid
mapscout grid "Thaltej, Ahmedabad"
mapscout grid "Pune" --radius-km 5 --geojson grid.geojsonShows how the area was matched, its size, the number of grid cells and the searches it would take. --geojson writes the area and cells to a file you can drop onto geojson.io to see them on a map.
mapscout check
mapscout check
mapscout check --query dentists --lat 23.05 --lng 72.51Runs one search (default: restaurants in central Surat) and reads one business page, then checks the fields still come through.
Prints PASS or FAIL per check, appends the result to logs/checks.log in the data folder, and exits with code 1 on any failure.
Run it before long jobs, or daily (see Maintenance).
Options reference
Options for mapscout scrape. Options that switch something off are written --no-….
What to search
| Option | Default | What it does |
|---|---|---|
AREA | asked | Area to cover, e.g. "Thaltej, Ahmedabad". |
-c, --categories | asked | Business types, comma-separated. |
--categories-file | A text file with one business type per line (lines starting with # are ignored). | |
--radius-km | Cover this many km around the area's centre instead of its boundary. | |
--all-results | off | Keep every business Google returns, not only the kind searched for. |
--include-outside | off | Also keep businesses found just outside the area. |
How much to collect
| Option | Default | What it does |
|---|---|---|
--max-places | no limit | Stop after this many businesses. Searching still covers the whole area; the rest go on the "Not read yet" sheet. |
--quick | off | Skip business pages and use the details that come with search results. Several times faster; no weekly hours or reviews. |
--reviews | 50 | Most reviews per business. Google shows signed-out visitors about 5, so this is an upper bound. 0 skips reviews. |
--review-sort | newest | newest, relevant, highest or lowest, when Google offers sorting. |
--no-enrich | Skip the website check (emails and social profiles). |
Speed and reuse
| Option | Default | What it does |
|---|---|---|
--concurrency | 3 | Browser tabs working on Google at once (1–10). More is not much faster and gets blocked sooner. |
--reuse-days | 30 | Copy a business another job read within this many days instead of opening it again. |
--refresh | off | Read every business again, even if read recently. |
--keep-overlapping | off | Search every term fully, even one that mostly repeats an earlier term. |
--no-early-stop | Scroll every result list to the end, including far-away filler. |
Grid
| Option | Default | What it does |
|---|---|---|
--cell-km | 3 | Size of the starting grid cells. |
--max-depth | 3 | How many times a dense cell may be split in four (3 means down to 0.375 km). |
Running and output
| Option | Default | What it does |
|---|---|---|
--block-wait-hours | 2 | If Google blocks, wait this long and continue, up to 3 times. 0 stops instead. Also on resume. |
--out | data folder | Where to write the Excel file. |
--data-dir | ~/mapscout-data | The data folder. Same as setting MAPSCOUT_DATA_DIR. Works on every command that uses saved data. |
-v, --verbose | off | Show every search and business as it happens (always written to the job's log). |
--show-browser | off | Show the browser window, for debugging. |
Common tasks
Try a small run first
mapscout scrape "Adajan, Surat" -c dentists --max-places 20Get the full list after a trial run
Run it again without the limit. The businesses already read are reused, not opened again.
mapscout scrape "Adajan, Surat" -c dentistsA fast survey: names, phones and websites only
mapscout scrape "Vesu, Surat" -c "software companies" --quickSeveral related kinds of business
mapscout scrape "Piplod, Surat" -c "software companies,web design companies,mobile app developers"A term whose results mostly repeat an earlier term is dropped automatically after 3 searches; the run report says so.
Everything near a landmark
mapscout scrape "Iscon Cross Road, Ahmedabad" -c cafes --radius-km 2Continue a stopped run
mapscout jobs
mapscout resume 12Refresh old data
mapscout scrape "Thaltej, Ahmedabad" -c "software companies" --refreshKeep the data somewhere else
# Windows PowerShell, for this terminal session
$env:MAPSCOUT_DATA_DIR = "D:\mapscout-data"
# macOS / Linux
export MAPSCOUT_DATA_DIR=~/Documents/mapscout-dataOutput
The data folder
All results go to one folder, whichever folder you run from, so every job can be found again:
- Windows:
C:\Users\<you>\mapscout-data - macOS / Linux:
~/mapscout-data
| Inside | What it is |
|---|---|
exports/<area>_job<N>.xlsx | The Excel workbooks. |
mapscout.db | The SQLite database: every job, search and business. Needed for resume, report, export and reuse. |
logs/job_<N>.log | Everything a job did, with timings and errors. |
logs/checks.log | One line per mapscout check. |
snapshots/job_<N>/ | Raw data of each business page read, used by reparse. |
It lives outside the project folder so scraped data never ends up in git. Deleting it removes all jobs and results.
Excel sheets
| Sheet | Contents |
|---|---|
| Summary | The search (area, matched boundary, business types), counts (found, on the Businesses sheet, not read, with a phone / website / email, reviews collected) and the run report. |
| Businesses | One row per business, every field. Links are clickable; the header row is frozen and filterable. |
| Reviews | Business, place ID, rating, date as shown and approximate date, review text, owner's reply and its date, review ID. No reviewer names. |
| Attributes | One row per attribute: business, place ID, section (e.g. Accessibility), attribute, Yes/No. |
| Not read yet | Businesses found but not read (the --max-places limit, a stopped run, or errors), with the reason and what the search results already said: category, rating, phone, website, address, Maps link. mapscout resume reads them. |
Business columns
| Group | Columns |
|---|---|
| Identity | Name, Primary category, All categories, Place ID, CID, Google Maps URL |
| Contact | Phone, Phone (international), Website, Emails, Emails (domain has no mail server) |
| Social | Facebook, Instagram, LinkedIn, X / Twitter, YouTube, TikTok, WhatsApp (found on the business's website) |
| Location | Address, Street, Neighborhood, City, Postal code, State, Country, Plus code, Latitude, Longitude, Inside area |
| Reputation | Rating, Review count, 5★ … 1★ reviews, Review topics (e.g. "ambience (413)"), Reviews collected, Photos |
| Hours | Monday … Sunday (only today's day in Google's limited view) |
| About | Status (operational / temporarily or permanently closed), Claimed by owner, Price, Description, Google summary, Highlights, Booking / ordering, Menu |
| Where it came from | Data source (own page, search results only, or reused from job N), Last updated, Google view (full / limited), Not read (parts of the page that failed), Found via (the search terms that found it) |
Emails lists addresses whose domain has a mail server, or couldn't be checked. Addresses whose domain has no mail server at all are kept separately, since mail to them would bounce.
How it works
- Area. The name is looked up on OpenStreetMap. Its official boundary is used when one exists, otherwise its bounding box. Some neighbourhoods aren't mapped as areas (Vesu, Surat, for example); then it covers 2 km around the closest match and says so.
- Grid. One Google Maps search shows at most 120 results, so the area is covered with square cells (3 km to start) and each business type is searched in each cell. A cell is split in four when its search is full of local results: 100+ results, almost none far away, and at least 30% inside the cell. Otherwise it's done.
- Filter. Google treats a search as keywords, so "tech companies" also returns builders and hostels with "tech" in their names. A result is kept only if one of its Google categories names the search term ("restaurants" → Chinese restaurant), or is the category Google returned most for that term ("tech companies" → Software company). Everything else is skipped before its page is opened, and listed in the run report.
- Details. Each search result already carries most of the business's record. Its own page then adds weekly hours, reviews and the rest. If a page keeps failing, the search-result record is used, so the business isn't lost.
- Websites. Each business website, plus up to two contact or about pages, is checked for emails (including Cloudflare-hidden ones) and social profiles, and each email's domain for a mail server.
- Excel. Written at the end, and also while waiting out a block.
In parallel. Each browser tab takes whatever Google work is next: a business page if one is waiting, otherwise a search. Website checks don't touch Google and run alongside. The number of tabs talking to Google stays at --concurrency.
Avoiding repeat work. A search stops scrolling once the rest of the list is far-away filler. A business another job read in the last 30 days is copied. A term is dropped if, after 3 searches and 20 businesses, fewer than 20% of its results are new.
Always resumable. Every search and business is a row in SQLite with a status, saved as soon as it's done.
What Google limits
These come from Google, not from this tool:
- Full view or limited view. Google decides per browser session whether an automated, signed-out browser gets the full page or a "limited view". The limited view has no reviews, no review count, no price and only today's opening hours. The Google view column shows which each business got.
- About 5 reviews per business. Even in the full view, signed-out visitors see 3–5 reviews. The scraper never signs in.
- Speed. One connection can only search so fast. With 3 tabs: a search takes about 20 s, a business page about 4 s.
A neighbourhood with one business type usually takes 10–40 minutes;
--quickis several times faster. - Blocks. Too much traffic brings Google's "unusual traffic" page. The run then waits and continues; there is no CAPTCHA solving.
Troubleshooting
Click a problem to see the fix. Type to filter:
Installing
"mapscout is not recognized" / "command not found" install
The command isn't on your PATH yet.
- Open a new terminal: PATH changes only reach new ones.
- Check it's installed:
uv tool listshould showmapscoutwithmapscout. If not, runuv tool install --editable .in the project folder. - If it's installed but still not found, run
uv tool update-shelland open a new terminal. - Meanwhile,
uv run mapscout …from the project folder always works.
It works in one terminal but not in another install
Installs made from inside a sandboxed app terminal (some IDE or desktop-app terminals) can land in a private copy of your user folder that other terminals don't see. Reinstall from a normal terminal (PowerShell / Windows Terminal / Terminal.app):
uv tool install --editable D:\path\to\mapscout --reinstall"Executable doesn't exist" or "Looks like Playwright was just installed" install browser
The Chromium browser Playwright drives isn't downloaded yet:
uv run playwright install chromiumOn Linux, add --with-deps if it complains about missing libraries.
"uv is not recognized" install
Close and reopen the terminal after installing uv. If it's still missing, re-run the uv installer from Installation.
PowerShell refuses to run the uv installer script install windows
Use the exact command from Installation: -ExecutionPolicy ByPass applies to that one command only and changes nothing else.
Choosing the area
"OpenStreetMap has no match for …" area
Write it as "<area>, <city>" and add the state if needed: "Vesu, Surat, Gujarat". Check spelling as OpenStreetMap writes it.
When prompted, it simply asks again. Alternatively pick a nearby landmark with --radius-km 2.
The matched area is wrong, too big or too small area
The plan line shows what was matched and its size; mapscout grid "<area>" shows it without searching.
- Too big (a district instead of a neighbourhood): be more specific,
"Thaltej, Ahmedabad". - "covers 2 km around the closest match": OpenStreetMap knows the name only as a point. Set the size yourself with
--radius-km. - Use
--geojson grid.geojsonongridand open it on geojson.io to see the exact area.
Results
Businesses of the wrong kind in the results results filter
Only results whose Google category matches your term are kept. If you still see others:
- Check you didn't use
--all-results. - A business can have several categories; one of them matched (see All categories).
- A vague term is read through its main category, shown in the job's log ("keeping businesses listed as …"). Use a more specific term.
Businesses I expected are missing results filter
- The run report's Skipped: other kinds of business line lists the categories left out. Google files some under a different name ("Dental clinic" rather than "Dentist"): add that as a second term,
-c "dentists,dental clinics". - Check the Not read yet sheet: businesses found but not read because of
--max-places, a stopped run or errors.mapscout resume <job>reads them. - A business just outside the boundary is left out; use
--include-outsideor--radius-km. - A term may have been dropped as repetitive (run report). Use
--keep-overlapping.
No reviews, empty review count, or only today's hours results google
Google served those pages in its limited view (see the Google view column). That's Google's choice per session; the scraper doesn't work around it.
A later run may get the full view: mapscout scrape … --refresh reads the pages again.
Only about 5 reviews per business although I asked for 50 results google
That's all Google shows signed-out visitors. The scraper never signs in, so --reviews is an upper bound.
Few or no emails results websites
- Many small businesses list no website, or only a Facebook / Instagram page (those are recorded as social profiles, not crawled).
- Many websites show no email at all, only a contact form.
- Check the run wasn't started with
--no-enrich. - Emails whose domain has no mail server are in their own column, Emails (domain has no mail server).
Price column is empty results google
The price range comes from the search result cards, and Google shows it mainly for restaurants and only in the full view.
"Data source" says "Search results only" results
Either --quick was used, or the business's own page failed on every try and the details that came with the search results were used instead. mapscout resume after mapscout check passes will try the page again.
Running
It's slow speed
mapscout report <job>shows where the time went: searches per minute, slow-downs, failures.- Fewer, more specific business types: every type is a full set of searches.
--quickskips business pages entirely.--max-placesfor a trial; the rerun reuses what was read.- Raising
--concurrencyhelps little and gets blocked sooner.
"Google asked this connection to slow down" / job status "blocked" blocked google
Google showed its "unusual traffic" page. The run saves the Excel file so far, waits 2 hours (--block-wait-hours) and continues, up to 3 times. Nothing is lost.
- If it's still blocked, wait a few hours and run
mapscout resume <job>. - Next time use
--concurrency 2and fewer business types per run. - Don't run several jobs at once from the same connection.
"Requests are failing; pausing 1.5x longer between them" warning speed
Not an error: requests failed, so the scraper slowed down automatically. It speeds back up after successes. Many of these in a row usually precede a block; consider stopping and resuming later.
"Search failed …" or "Place page failed … Timeout" warnings warning
A page didn't load in time. Each item is retried (3 tries by default); a business whose page never loads is kept with its search-result details.
Occasional ones are normal. If most fail, check your connection and run mapscout check.
"couldn't read reviews / weekly hours … keeping the rest" warning
One part of a business page didn't work (something covered a button, for example). The business is kept; the Not read column names the missing part.
I pressed Ctrl+C, or the computer went to sleep resume
Everything done so far is saved. Continue with mapscout resume <job> (the job number is in mapscout jobs). Keep the computer awake for long runs.
A cookie consent page appears (EU) google
It's handled automatically by choosing "Reject all". If it keeps appearing, run once with --show-browser to see what's shown.
mapscout check fails check google
Google probably changed its pages. Don't start long runs until it passes. See Maintenance for where to fix it; a failure can also be temporary (connection, block), so try again later first.
Output
"Permission denied" when writing the Excel file excel windows
The file is open in Excel, which locks it on Windows. Close it, then write it again: mapscout export <job>. Nothing is lost; the data is in the database.
I can't find my Excel file excel
The path is printed at the end of every run. Files are in exports inside the data folder, named <area>_job<N>.xlsx.
mapscout jobs doesn't show an old job data
Jobs live in the data folder's database. If MAPSCOUT_DATA_DIR or --data-dir pointed elsewhere at the time, use the same setting again.
Strange characters (₹, ?) in the terminal windows
An old console code page. Use Windows Terminal, or run $env:PYTHONIOENCODING = "utf-8" first. The Excel file is unaffected.
Start over and delete everything data
Delete the data folder. That removes all jobs, results and logs; the program itself is untouched.
FAQ
Do I need a Google account or API key?
No. It uses Google Maps like a signed-out visitor, and never signs in.
Can I scrape a whole city?
It's built for an area within a city. A whole city with many business types means thousands of searches and many hours; check with mapscout grid first.
Why not more browser tabs to go faster?
Google limits how fast one connection can search. More tabs mostly means being blocked sooner.
Does it run on a server?
Yes, it runs headless by default. Install the same way; on Linux use playwright install --with-deps chromium.
Where do I report a bug?
Open an issue on GitHub with the command you ran and the relevant part of logs/job_<N>.log.
Maintenance
Daily health check
Windows, every morning at 9 (run once in your own terminal):
schtasks /Create /TN "mapscout health check" /TR "%USERPROFILE%\.local\bin\mapscout.exe check" /SC DAILY /ST 09:00Remove it with schtasks /Delete /TN "mapscout health check". Results go to logs/checks.log.
macOS / Linux (crontab -e):
0 9 * * * $HOME/.local/bin/mapscout checkWhen Google changes its pages
- Everything the scraper looks for on a page:
src/mapscout/selectors.py. - Field positions in Google's data:
parse_place_arrayinsrc/mapscout/parsing.py. - After fixing the parser, update existing jobs without searching again:
mapscout reparse <job>, thenmapscout export <job>. - Run
uv run pytest: the tests use real captured data and don't touch the internet.
Project layout
| File | What it does |
|---|---|
cli.py | The mapscout commands and prompts |
pipeline.py | Runs a job: searches, business pages and website checks in parallel |
discovery.py | One Google Maps search, scrolled to the end |
detail.py | One business page: hours, reviews and the rest |
categories.py | Which results are the kind of business searched for |
parsing.py | Reading fields out of Google's data |
selectors.py | Everything looked for on Google's pages |
enrich.py | Emails and social profiles from business websites |
geo.py | Area lookup and the search grid |
store.py | The SQLite database and job queue |
export.py, report.py | The Excel workbook and the run report |
check.py | The health check |
Responsible use
- Google's Terms of Service do not allow automated collection from Google Maps. The official, paid alternative is the Places API.
- Business data can include personal details, such as a sole trader's phone number or email. Follow the privacy and anti-spam laws that apply to you (for example GDPR in the EU or the DPDP Act in India) before contacting anyone.
- Reviewer names and profile links are never collected.
- Keep the defaults polite: a few tabs, with pauses between requests.
License
MIT: free to use, copy, modify and distribute, including commercially, as long as the license notice is kept. The software comes with no warranty. See the LICENSE file.