Push Feeds & RSS

Superfeedr is a Feed API designed to make handling RSS, Atom, or JSON feeds easier. Aimed at publishers as well as subscribers, Superfeeds helps manage and streamline the notification process.

Feedfetcher is how Google grabs RSS or Atom feeds for Google Play Newsstand and PubSubHubbub. Feedfetcher collects and periodically refreshes these user-initiated feeds, but does not index them in Blog Search or Google's other search services (feeds appear in our search results only if they've been crawled by Googlebot). Feedfetcher behaves as a direct agent of the human user, not as a robot, so it ignores robots.txt entries.

APIs-Google is the user agent used by Google APIs to deliver push notification messages. Application developers can request these notifications to avoid the need for continually polling Google's servers to find out if the resources they are interested in have changed. APIs-Google sends each push notification using an HTTPS POST request. If you subscribe to this API notification service you should allow this crawler.

FeedBurner is a web feed management provider launched in 2004. It provides custom RSS feeds and management tools for bloggers, podcasters, and other web-based content publishers. Google acquired FeedBurner in 2007. It can be used to monitor your RSS feeds, if you don't publish and use RSS feeds, you are safe to block it.

Open source go crawler

Free RSS reader and content aggregator for newsletter content, YouTube, Blogs Podcast and news and magazine content all from one reader