usp package

Submodules

usp.exceptions module

Exceptions used by the sitemap parser.

exception usp.exceptions.GunzipException[source]

Bases: Exception

gunzip() exception.

exception usp.exceptions.SitemapException[source]

Bases: Exception

Problem due to which we can’t run further, e.g. wrong input parameters.

exception usp.exceptions.SitemapXMLParsingException[source]

Bases: Exception

XML parsing exception to be handled gracefully.

exception usp.exceptions.StripURLToHomepageException[source]

Bases: Exception

strip_url_to_homepage() exception.

usp.tree module

Helpers to generate a sitemap tree.

usp.tree.sitemap_tree_for_homepage(homepage_url: str, web_client: Optional[usp.web_client.abstract_client.AbstractWebClient] = None) → usp.objects.sitemap.AbstractSitemap[source]

Using a homepage URL, fetch the tree of sitemaps and pages listed in them.

Parameters:
  • homepage_url – Homepage URL of a website to fetch the sitemap tree for, e.g. “http://www.example.com/”.
  • web_client – Web client implementation to use for fetching sitemaps.
Returns:

Root sitemap object of the fetched sitemap tree.

Module contents