uninews.fetchers.base.Fetcher Class Reference

Inheritance diagram for uninews.fetchers.base.Fetcher:

Public Member Functions
	__init_subclass__ (cls, **kwargs)
	__init__ (self, str publisher, str category, dict config)
str	__repr__ (self)
list[dict]	fetch_raw (self)
list[Article]	fetch (self)
requests.Response	request (self, str url)
str	get_html (self, str url)

Public Attributes
	publisher
	category
	config

Static Public Attributes
ClassVar	fetcher_type = ""
tuple	required_keys = ("label", "url")
dict	registry = {}

Detailed Description

One page or feed to fetch, configured by a JSON entry.

Subclasses set `fetcher_type` (the JSON "type" value) and implement fetch_raw().
Defining a subclass registers it automatically.

Constructor & Destructor Documentation

__init__()

uninews.fetchers.base.Fetcher.__init__	(		self,
		str	publisher,
		str	category,
		dict	config
	)

Reimplemented in uninews.fetchers.html.HtmlFetcher.

Member Function Documentation

__init_subclass__()

uninews.fetchers.base.Fetcher.__init_subclass__	(		cls,
		**	kwargs
	)

__repr__()

str uninews.fetchers.base.Fetcher.__repr__

(

self

)

fetch()

list[Article] uninews.fetchers.base.Fetcher.fetch

(

self

)

Fetch, clean, and deduplicate. This is what the app calls.

fetch_raw()

list[dict] uninews.fetchers.base.Fetcher.fetch_raw

(

self

)

Return dicts with 'title' and 'link'.

Optional keys: 'published', 'summary', 'image_url'.
Links and image URLs may be relative.

Reimplemented in uninews.fetchers.html.HtmlFetcher, and uninews.fetchers.rss.RssFetcher.

get_html()

str uninews.fetchers.base.Fetcher.get_html	(		self,
		str	url
	)

request()

requests.Response uninews.fetchers.base.Fetcher.request	(		self,
		str	url
	)

Member Data Documentation

category

uninews.fetchers.base.Fetcher.category

config

uninews.fetchers.base.Fetcher.config

fetcher_type

ClassVar uninews.fetchers.base.Fetcher.fetcher_type = ""

static

publisher

uninews.fetchers.base.Fetcher.publisher

registry

dict uninews.fetchers.base.Fetcher.registry = {}

static

required_keys

tuple uninews.fetchers.base.Fetcher.required_keys = ("label", "url")

static

---

The documentation for this class was generated from the following file:

• src/uninews/fetchers/base.py