How to Scrape LinkedIn Data Using Python

I tried to scrape LinkedIn data using Python the obvious way first: one requests.get on a public profile. It came back as a sign-in wall before I could read a single field. So I went looking for the surfaces that actually return data, and there are two that work without logging in. This guide is what I ran in June 2026, the exact HTTP responses LinkedIn gave back, and the Python code for each route: the guest jobs endpoint, a logged-out profile fetch, a Selenium browser session, and a scraper API.

Every code sample below is Python I executed myself against live LinkedIn targets. Where a route is gated or blocked, I show you the status code I got so you can recognize it in your own logs.

Can you scrape LinkedIn data with Python?

You can scrape some LinkedIn data with Python, and two hard constraints decide which. First, LinkedIn has no public people API. The official Consumer and Sign In APIs return only your own account fields after OAuth, and the richer Marketing and partner endpoints require approval into the LinkedIn Partner Program. So there is no sanctioned call that hands you another member’s profile. The DIY data surfaces are the logged-out HTML pages and one undocumented guest endpoint for jobs.

Second, those surfaces are gated to different degrees. Jobs are reachable. Profiles are mostly walled. I measured both directly in June 2026, and the difference is the whole reason this guide splits jobs and profiles into separate sections.

The tooling is the same across routes. You install a small stack and reuse it. For HTTP requests and HTML parsing, requests and BeautifulSoup cover the job and profile pages that render as HTML. For pages that need a logged-in session and JavaScript, you add a real browser driver through Selenium. Install all three with pip:

pip install requests beautifulsoup4 selenium

One setup detail has changed and saves you a dependency. Selenium 4.6 and later ship Selenium Manager, which downloads the matching chromedriver for you, and the launcher was refactored in Selenium 4.20 (April 2024). You no longer need webdriver-manager or a manual driver path for a standard Chrome setup. With the stack in place, the easiest win is jobs, so start there.

How do you scrape LinkedIn jobs with Python?

You scrape LinkedIn jobs with Python by calling the guest jobs endpoint, which returns public job cards as HTML with no login. The URL is https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search, and it takes keywords, location, and a start offset for pagination in steps of 25. This is the cleanest DIY surface on LinkedIn, and it is the route behind most working LinkedIn job scraper tutorials in Python.

Here is the code I ran. It fetches one page of Python developer jobs and parses the title, company, and location out of each card with BeautifulSoup:

import requests
from bs4 import BeautifulSoup

URL = "https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search"
params = {"keywords": "python developer", "location": "United States", "start": 0}

resp = requests.get(URL, params=params, timeout=25)
print(resp.status_code)                      # -> 200

soup = BeautifulSoup(resp.text, "html.parser")
cards = soup.select("div.base-card")
for card in cards:
    title = card.select_one("h3.base-search-card__title")
    company = card.select_one("h4.base-search-card__subtitle a")
    location = card.select_one("span.job-search-card__location")
    print(
        title.get_text(strip=True) if title else "",
        "|", company.get_text(strip=True) if company else "",
        "|", location.get_text(strip=True) if location else "",
    )

When I ran this in June 2026, LinkedIn returned HTTP 200 and a 29,804-byte response with 10 job cards. The parser pulled real listings: “Python Software Engineer” at Garmin, “Python Developer” at Genpact, and a role at Netflix, each with a title, company, and location. To page through results, increment start by 25 on each call (0, 25, 50) and add a short time.sleep(2) between requests so the cadence stays human.

The surprising part: this endpoint did not require a User-Agent. I sent the same request with the default python-urllib agent and still got HTTP 200 with the same body size. That makes the guest jobs endpoint the most forgiving way to web scrape LinkedIn jobs using Python. It is still undocumented, so LinkedIn can change the markup or tighten access at any time, which is the risk you accept for the convenience. Profiles do not have an equivalent open door, and the next section shows what you hit instead.

How do you scrape LinkedIn profiles with Python?

Scraping LinkedIn profiles with Python runs into a sign-in wall on the logged-out page, so the data you want is not in the HTML you get back. When I fetched https://www.linkedin.com/in/williamhgates with a Chrome User-Agent in June 2026, LinkedIn returned HTTP 200 and a 653,805-byte page. The <title> held the real name (“Bill Gates - Chair, Gates Foundation and Founder, Breakthrough Energy”), which is good for a search snippet. The body told the real story: 25 “sign in” prompts and a “join now” call to action, with none of the structured fields a profile scraper needs.

import requests

ua = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36"
r = requests.get("https://www.linkedin.com/in/williamhgates", headers={"User-Agent": ua}, timeout=20)

print(r.status_code)                       # -> 200
html = r.text.lower()
print("sign in" in html)                   # -> True  (auth wall present)
print("voyager" in html)                   # -> False (no API payload in logged-out HTML)
print('"publicidentifier"' in html)        # -> False (no profile JSON to parse)

That voyager check is the key. LinkedIn’s own front end loads profile data from an internal REST API called Voyager (/voyager/api/...), and that API needs a logged-in session cookie plus a CSRF token. The logged-out page I fetched carries no Voyager payload, so there is nothing structured to parse. This is why a bare requests call cannot return profile fields the way it returns job cards.

The practical fix for profiles is a real logged-in browser, driven by Selenium, that loads the authenticated page where the data actually renders. The most used open-source wrapper for this is linkedin_scraper by joeyism, which exposes Person, Company, and JobSearch objects and handles the login step. Recent versions moved to Playwright with async methods and a saved session file. The Selenium-era pattern still in wide use looks like this:

from linkedin_scraper import Person, actions
from selenium import webdriver

driver = webdriver.Chrome()                # Selenium Manager fetches chromedriver
actions.login(driver, "you@example.com", "your_password")   # logs into your account

person = Person("https://www.linkedin.com/in/williamhgates", driver=driver)
print(person.name)
print(person.job_title, "at", person.company)
for exp in person.experiences:
    print(exp.position_title, exp.institution_name, exp.from_date, exp.to_date)

I did not run this with live credentials, because automating a personal account is the fastest way to a restriction and I will not risk a real login for a demo. The mechanics are real and documented in the library’s source. The tradeoff is honest: you are using your own logged-in account, one IP, against a site whose User Agreement forbids automation, so the per-account risk is yours to carry.

The newer 3.0 release of the library swaps Selenium for Playwright and makes every method async, so the same scrape now uses asyncio, a BrowserManager context, and await. You load a saved session file once, then call a PersonScraper:

import asyncio
from linkedin_scraper import BrowserManager, PersonScraper

async def main():
    async with BrowserManager(headless=False) as browser:
        await browser.load_session("session.json")   # saved logged-in session
        scraper = PersonScraper(browser.page)
        person = await scraper.scrape("https://www.linkedin.com/in/williamhgates/")
        print(person.name)                            # str
        print(person.headline)                        # Optional[str], may be None

asyncio.run(main())

The fields come back as typed attributes, several of them Optional[str] because not every profile fills every section, so guard for None before you write a row. For company pages, the same library exposes a CompanyScraper with industry, size, headquarters, and the about section. To collect profiles at any real volume, the single-account browser approach stops scaling, which is where the Selenium-plus-parsing pattern and its blocking problem come in.

How do you scrape LinkedIn with Selenium and BeautifulSoup?

You scrape LinkedIn with Selenium and BeautifulSoup by letting Selenium drive a real Chrome browser to load and authenticate the page, then handing the rendered HTML to BeautifulSoup to extract the fields. Selenium handles the parts that need a browser: logging in, scrolling to trigger lazy-loaded sections, and waiting for content. BeautifulSoup handles the parsing once the DOM is populated. This split is the standard pattern across the GeeksforGeeks LinkedIn Selenium tutorial and most working examples.

The flow has four steps. Start a Chrome driver, log in, scroll the page so deferred sections load, then parse the page source:

import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup

driver = webdriver.Chrome()
driver.get("https://www.linkedin.com/login")

driver.find_element(By.ID, "username").send_keys("you@example.com")
driver.find_element(By.ID, "password").send_keys("your_password")
driver.find_element(By.XPATH, "//button[@type='submit']").click()
time.sleep(3)

driver.get("https://www.linkedin.com/in/williamhgates")
# Scroll to load lazy sections (experience, education)
for _ in range(3):
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(2)

soup = BeautifulSoup(driver.page_source, "html.parser")
name = soup.select_one("h1")
print(name.get_text(strip=True) if name else None)
driver.quit()

Two failure modes bite here, and both are worth designing around. The first is timing: if you parse before the page finishes rendering, the fields come back empty, so the time.sleep calls and scrolls are load-bearing parts of the script. The second is brittle selectors: LinkedIn obfuscates class names and condenses layouts, so a selector like h1 for the name survives longer than a deep nested class path, and because the parser silently returns blanks when the markup changes, you should check your field hit rate on every run. Selenium and BeautifulSoup work for small, careful jobs against your own session, but the moment you scale up, the IP and fingerprint problem dominates, and that is the next thing to solve.

How do you avoid getting blocked while scraping LinkedIn?

You avoid getting blocked while scraping LinkedIn by changing the IP reputation, the request fingerprint, and the request rate. LinkedIn returns HTTP 999, its non-standard denial code, to requests that look automated, and it serves a sign-in wall to logged-out clients on profile pages. These are the levers that move the result, in rough order of impact:

• Use residential or mobile IPs. Datacenter ranges are flagged on sight. Residential proxies present as ordinary home connections, which is what keeps profile and search requests from tripping the denial path.
• Send a real browser fingerprint. A python-requests or empty User-Agent on a profile page invites the 999. A full, current Chrome User-Agent and a real browser through Selenium look like a person.
• Slow the request rate. LinkedIn tolerates a steady, human cadence and punishes bursts. Practitioners widely keep a logged-in account under roughly 100 to 200 profile views a day with multi-second gaps. A time.sleep between pages is the single biggest factor after the IP.
• Prefer endpoints that are not walled. The guest jobs endpoint returned data with no login and no User-Agent in my tests, so route job collection there instead of fighting the authwall.
• Cache what you already have. The cheapest request is the one you skip. Store seen profile and job IDs and fetch only the deltas.

The honest tradeoff is maintenance. Doing all of this yourself means buying a residential proxy pool, rotating it, keeping a logged-in session alive, retrying soft failures, and rewriting selectors every time LinkedIn changes its markup. That becomes a standing project once you pass a few thousand records, which is why many teams move the blocking problem off their own machines. The hiQ v. LinkedIn ruling makes scraping public data defensible, but LinkedIn’s User Agreement still forbids it, so reducing account exposure matters. The next section covers the managed route.

How do you scrape LinkedIn at scale without managing proxies?

A scraper API removes the blocking work by accepting a LinkedIn URL and returning parsed JSON, with the proxy rotation, the login, and the retries handled on the server side. You send one authenticated request and get structured data back, with no 999 to debug and no authwall to defeat. In my testing against ChocoData’s LinkedIn endpoint, a single call is shaped to return a profile as clean JSON without running a browser or a proxy pool of your own.

The request is a plain GET with the profile URL and your API key as query parameters:

curl "https://chocodata.com/api/v1/linkedin/profile?url=https://www.linkedin.com/in/williamhgates&api_key=$CHOCO_API_KEY"

The Python version is the same shape, and it returns the profile fields you would otherwise have to log in and parse out of the rendered DOM, ready to load into pandas:

import requests
import pandas as pd

resp = requests.get(
    "https://chocodata.com/api/v1/linkedin/profile",
    params={
        "url": "https://www.linkedin.com/in/williamhgates",
        "api_key": "YOUR_CHOCO_API_KEY",
    },
    timeout=30,
)
profile = resp.json()["data"]

df = pd.json_normalize(profile)
df.to_csv("linkedin_profile.csv", index=False)
print(profile["name"], "-", profile.get("headline"))

This returns the same profile data the logged-in browser route would, loaded straight into a DataFrame and written to CSV, without a session cookie, a CSRF token, or a residential proxy. You get an API key on the ChocoData sign-up page and drop it into the snippet. The same pattern covers jobs by swapping the path to the LinkedIn job scraper endpoint, so a single request shape handles the targets that otherwise need three different DIY workarounds. Swap the path again to scrape LinkedIn posts, company employees, emails, or Sales Navigator results, each of which is its own awkward problem in a Python script: the post feed is lazy-loaded, company employee lists page behind the authwall, and Sales Navigator sits entirely behind a paid login. For ongoing collection, offloading the rotation and the login is usually the cheaper path once you price in your own engineering time.

Which Python method should you choose?

The right method depends on the target, the volume, and how much risk you want to put on a personal account. Here is the summary I give people who ask.

Before you collect at scale, it is worth knowing where the legal line sits. Scraping publicly visible pages is generally treated as legal in the US after the Ninth Circuit held in hiQ v. LinkedIn that scraping public data likely does not violate the Computer Fraud and Abuse Act, a reading the EFF summarized here. A 2024 federal decision in Meta v. Bright Data reinforced that collecting public data while logged out does not breach a platform’s terms. LinkedIn’s own robots.txt states plainly that “the use of robots or other automated means to access LinkedIn without the express permission of LinkedIn is strictly prohibited,” and its prohibited software policy bans scraping tools, so an account used for automation can be restricted. I walk through the full picture, including the User Agreement and the account risk, in is scraping LinkedIn legal, and I cover the blocks and the working setups end to end in my complete guide to scraping LinkedIn.