Apify Cron Jobs: Schedule Scrapers to Run Automatically

Quick answer: how to schedule an Apify scraper?

Go to Apify Console > Schedules > Create new schedule. Select the Actor, configure the input, set the cron expression (e.g., "0 8 * * *" for daily at 8 AM), and save. The scraper runs automatically on schedule. Results are stored in datasets and can trigger webhooks for downstream automation.

Cron expressions explained

Apify uses standard cron expressions. Here is the format and common patterns:

• Format: minute hour day-of-month month day-of-week (5 fields)
• Every day at 8 AM: 0 8 * * *
• Every 6 hours: 0 */6 * * *
• Every hour: 0 * * * *
• Every Monday at 9 AM: 0 9 * * 1
• Every weekday at 8 AM: 0 8 * * 1-5
• First of every month at midnight: 0 0 1 * *
• Every 15 minutes: */15 * * * *
• Twice daily (8 AM and 8 PM): 0 8,20 * * *

Setting up a schedule via Apify Console

Here is the step-by-step process:

1. Go to Apify Console > Schedules > Create new — Go to Apify Console > Schedules > Create new
2. Name your schedule (e.g., "Daily Competitor Price Monitor") — Name your schedule (e.g., "Daily Competitor Price Monitor")
3. Select the Actor to run (e.g., Google Shopping Scraper) — Select the Actor to run (e.g., Google Shopping Scraper)
4. Configure the Actor input (search terms, URLs, max results) — Configure the Actor input (search terms, URLs, max results)
5. Set the cron expression (e.g., 0 8 * * * for daily at 8 AM) — Set the cron expression (e.g., 0 8 * * * for daily at 8 AM)
6. Set timezone (choose your local timezone for consistency) — Set timezone (choose your local timezone for consistency)
7. Enable webhooks if you want notifications (see our webhooks guide) — Enable webhooks if you want notifications (see our webhooks guide)
8. Click "Create schedule" — the scraper will run automatically on schedule — Click "Create schedule" — the scraper will run automatically on schedule

Setting up a schedule via API

For programmatic control, use the Apify API:

• POST https://api.apify.com/v2/schedules?token={token}
• Body: {"name": "Daily Price Monitor", "actId": "{actorId}", "cronExpression": "0 8 * * *", "timezone": "America/New_York", "runInput": {"searchQueries": ["marketing tools"], "maxResults": 50}}
• Response includes the schedule ID for management
• Update: PUT https://api.apify.com/v2/schedules/{scheduleId}?token={token}
• Delete: DELETE https://api.apify.com/v2/schedules/{scheduleId}?token={token}
• List all schedules: GET https://api.apify.com/v2/schedules?token={token}

Best scheduling patterns for marketing use cases

Different use cases require different schedules:

• Competitor price monitoring: Daily at 8 AM. Catches overnight price changes. Use Google Shopping Scraper.
• Brand mention tracking: Every 6 hours. Catches mentions throughout the day. Use Twitter/X Scraper + Reddit Scraper.
• Lead generation: Weekly (Monday 8 AM). Fresh leads for the week. Use Google Maps Scraper.
• Review monitoring: Daily at 6 AM. Respond to new reviews early. Use Google Maps Scraper (reviews).
• Trend monitoring: Every 2 hours. Catch trending topics early. Use TikTok Scraper + Twitter Trends.
• SEO rank tracking: Daily at midnight. Track rankings consistently. Use Google Search Scraper.
• Stock availability: Every 30 minutes. Time-sensitive for high-demand products. Use Amazon Product Scraper.

Handling schedule failures

Scheduled runs can fail. Here is how to handle failures gracefully:

• Webhook on failure: Set up a webhook for ACTOR.RUN.FAILED to get Slack/email alerts when a scheduled run fails.
• Auto-retry: Apify automatically retries failed runs once. For critical scrapers, set up a manual retry via webhook.
• Max retries: Limit retries to 2-3 to avoid wasting compute units on persistently failing scrapers.
• Failure log: Check Apify Console > Runs for error logs. Common failures: target site down, anti-bot detection, input errors.
• Fallback Actor: If a site-specific Actor fails, have a generic scraper (Cheerio/Puppeteer) as a backup with simpler extraction.
• Monitoring dashboard: Use Apify's run history to track success rate over time. Below 90% success rate needs attention.

Cost optimization for scheduled scraping

Scheduled runs consume compute units. Here is how to optimize costs:

• Right-size max results: Do not scrape 1,000 results when you need 100. Set max results to your actual need.
• Off-peak scheduling: Schedule runs during off-peak hours (early morning) when target sites are less likely to rate-limit.
• Incremental scraping: Only scrape new data since the last run. Use date filters to avoid re-scraping old content.
• Consolidate Actors: If you scrape 5 sites, use 5 Actors in one schedule instead of 5 separate schedules. This reduces overhead.
• Monitor usage: Check Apify Console > Billing monthly. Identify expensive schedules and optimize or downgrade them.
• Free tier: Apify free tier includes $5/month of usage. That covers 1-2 daily small runs. Upgrade to $49/month for regular scraping.

Combining schedules with webhooks for full automation

Schedules + webhooks = fully automated data pipeline:

• Schedule triggers scraper at 8 AM daily
• Scraper runs and collects competitor pricing data
• Webhook fires on SUCCEEDED with dataset ID
• Webhook handler compares new prices with previous run
• If price dropped 5%+, send Slack alert
• Append all data to Google Sheet for historical tracking
• Weekly summary email with all changes
• Result: Zero manual intervention. Full automation from scrape to alert.