Auto-login & cookies

Use login and cookie workflows in Octoparse for websites that require authenticated sessions.

Some websites require login before data can be viewed. Octoparse can work with login-required pages when the task is configured to preserve the necessary session state.

Use login and cookie workflows when the target data is only visible after authentication and you have permission to access it.

When login setup is needed

Login or cookies may be required when:

• The website hides data behind an account
• Search results or detail pages require authentication
• The website shows different content after login
• A session expires during extraction
• Cloud runs fail because the login state is not available
• Cookies control region, language, or user-specific content

Typical workflow

1. Open the website

   Start from the login page or a page that requires authentication.

2. Use Browse Mode

   Interact with the page like a normal browser to complete login or reach the desired page state.

3. Save the session setup

   Configure the task so the required login or cookie state is available during extraction.

4. Test the task

   Run a sample to confirm the task can access the protected content.

5. Monitor expiration

   Recheck the task if the website logs users out or invalidates cookies.

Cookies and sessions

Cookies store information that helps a website recognize a browser session. They may include login state, preferences, region, language, or tracking information.

For scraping tasks, cookies matter because the website may show different content depending on whether the session is valid.

Common issues

Issue	Possible cause
Task returns login page	Session expired or login was not preserved
Works locally but fails in cloud	Cloud run does not have the same session state
Data differs between runs	Cookies, region, or account state changed
CAPTCHA appears after login	Website detected unusual session behavior
Task stops after some pages	Session expired mid-run

Best practices

• Use an account you are authorized to use.
• Test login-required tasks locally before cloud runs.
• Avoid rotating IPs during the same logged-in session.
• Re-authenticate when cookies expire.
• Monitor cloud logs for login redirects.
• Keep login steps as simple and stable as possible.

Only extract data you are permitted to access. Login access does not automatically mean the data may be collected or reused.