mini_buildd.util module

class mini_buildd.util.Versions

Bases: dict

Main software component’s version support (for compat code, informational)

has(component, version_string)
mini_buildd.util.http_endpoint(number=0)
mini_buildd.util.daemon()

Import daemon module and return Daemon singleton (sort-of dependency injection)

Use this where you need the models module but can’t do a ‘proper import’ – as django needs to be configured first.

mini_buildd.util.models()

Import and return models module (sort-of dependency injection)

Use this where you need the models module but can’t do a ‘proper import’ – as django needs to be configured first.

mini_buildd.util.NEWLINE = '\n'

For use in fstrings

mini_buildd.util.fopen(path, mode='r', **kwargs)

Text file open with our fixed char encoding (UTF-8)

UTF-8 may become default for open at some point, but not just yet.

See https://www.python.org/dev/peps/pep-0597/

class mini_buildd.util.PyCompat

Bases: object

Misc helpers to stay compatible with python <= 3.6 (see debian/control)

static removeprefix(s, prefix)

Attention

compat (python < 3.9): str.removeprefix

static shlex_join(split_command)

Attention

compat (python < 3.8): shlex.join

class mini_buildd.util.Rfc7807(status, detail=None)

Bases: object

to_json()
classmethod from_json(data)
exception mini_buildd.util.HTTPError(status, detail=None)

Bases: Exception

Public (HTTP) exception – raise this if the exception string is ok for user consumption

exception mini_buildd.util.HTTPOk(detail=None)

Bases: HTTPError

exception mini_buildd.util.HTTPNotFound(detail=None)

Bases: HTTPError

exception mini_buildd.util.HTTPBadRequest(detail=None)

Bases: HTTPError

exception mini_buildd.util.HTTPUnauthorized(detail=None)

Bases: HTTPError

exception mini_buildd.util.HTTPUnavailable(detail=None)

Bases: HTTPError

exception mini_buildd.util.HTTPInternal(detail=None)

Bases: HTTPError

mini_buildd.util.log_exception(log, message, exception, level=30)
mini_buildd.util.log_stack(log, level=20)

Log stack (for debugging)

mini_buildd.util.e2http(exception, status=HTTPStatus.INTERNAL_SERVER_ERROR)
mini_buildd.util.rrpes(func, *args, **kwargs)

Run func. On exception, return public error str

mini_buildd.util.check_program(path, deb=None)
mini_buildd.util.systemcert_workaround()

Set environment SSL_CERT_FILE to point to system’s certificate store (ca-certificates) when not using libssl3

Note

compat (< libssl3): python fails for (self-signed) certificate even though it’s enrolled with system’s ca-certificates?

You will get some error like certificate verify failed: self signed certificate.

This seems to occur with libssl1, while it seems fine with libssl3 (even though Debian Bug #805646 never got closed).

Check the package deps on libpythonX.Y-minimal to see if it was build against libssl1 or libssl3 (or mini-buildd’s log for warning message).

Simplest workaround is to set SSL_CERT_FILE environment, for example like so:

export SSL_CERT_FILE="/etc/ssl/certs/ca-certificates.crt"
mini_buildd.util.singularize(s)

Singularize some english nouns from plural

mini_buildd.util.esplit(s, sep)

Unlike python str.strip(sep), this delivers an empty list for the empty string

mini_buildd.util.uniq(iterable)

Make list unique

Sometimes, a list should be unique, but you don’t want to use set (for example, as it does not keep order or can’t be JSON-serialized).

There does not seem to be a simple builtin for this. Since python 3.7, dict is guaranteed to be ordered, so we can use this approach.

mini_buildd.util.mdget(d, keys, default=None)

Get nested value from multidimensional dict, return default if any ‘path part’ is missing

Rather use this instead of code like d.get(foo, {}).get(bar, {}).get(mykey, mydefaultvalue).

>>> d = {"foo": {"bar": {"mykey": "myvalue"}}}
>>> mdget(d, ["foo", "bar", "mykey"], "default_value")
'myvalue'
>>> mdget(d, ["fpp", "bar", "mykey"], "default_value")
'default_value'
>>> mdget(d, ["foo", "bsr", "mykey"], "default_value")
'default_value'
>>> mdget(d, ["foo", "bar", "mykex"], "default_value")
'default_value'
class mini_buildd.util.Datetime

Bases: object

Datetime tools – always use same/comparable (aware && UTC) datetime objects

EPOCH = datetime.datetime(1970, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)
SINCE_OPTIONS = ['1 hour ago', '1 day ago', '1 week ago', '1 month ago', '1 year ago', 'epoch']
classmethod now()

Shortcut to be used for all internal UTC stamps

classmethod from_stamp(stamp)
classmethod from_iso(isostr)
classmethod from_path(path)
classmethod timecode(stamp=None)

Timecode string from current timestamp

classmethod is_naive(date_obj)
classmethod check_aware(date_obj)
classmethod parse(date_string, default=None)

Use keyword ‘epoch’ for a big-bang timestamp. Otherwise most formats should work (like ‘2 weeks ago’)

class mini_buildd.util.Snake(name)

Bases: object

Case style conversion to (lowercase) snake

>>> Snake("CamelCase").from_camel()
'camel_case'
>>> Snake("Kebab-Case").from_kebab()
'kebab_case'
from_camel()
from_kebab()
class mini_buildd.util.Field(field)

Bases: object

Changes field name handling (custom prefix && (snake) name conversion)

CPREFIX = 'X-Mini-Buildd-'
class mini_buildd.util.CField(field)

Bases: Field

class mini_buildd.util.StopWatch

Bases: object

close()
delta()
to_json()
class mini_buildd.util.Singleton

Bases: type

destroy()
class mini_buildd.util.TmpDir(**kwargs)

Bases: object

Temporary dir class (use with with contextlib.closing()..., or as mixin class)

close()
mini_buildd.util.tmp_dir(**kwargs)

Temporary dir context manager (use with with)

mini_buildd.util.nop(*_args, **_kwargs)
mini_buildd.util.attempt(func, *args, retval_on_failure=None, **kwargs)

Run function, warn-log exception on error, but continue

>>> attempt(lambda x: x, "ypsilon")
'ypsilon'
>>> attempt(lambda x: x, "ypsilon", retval_on_failure="xylophon", unknown_arg="xanthippe")
'xylophon'
>>> attempt(lambda x: x/0, "ypsilon", retval_on_failure="xylophon")
'xylophon'
mini_buildd.util.measure(func, *args, **kwargs)
mini_buildd.util.strip_epoch(version_str)

Strip the epoch from a version string

mini_buildd.util.guess_default_dirchroot_backend(overlay, aufs)
mini_buildd.util.subst_placeholders(template, placeholders)

Substitute placeholders in string from a dict

>>> subst_placeholders("Repoversionstring: %IDENTITY%%CODEVERSION%", { "IDENTITY": "test", "CODEVERSION": "60" })
'Repoversionstring: test60'
class mini_buildd.util.Hash(path)

Bases: object

Shortcut to get hashsums from file

>>> Hash("test-data/unix.txt").md5()
'cc3d5ed5fda53dfa81ea6aa951d7e1fe'
>>> Hash("test-data/unix.txt").sha1()
'8c84f6f36dd2230d3e9c954fa436e5fda90b1957'
get(hash_type='md5')

Get any hash from file contents

md5()
sha1()
mini_buildd.util.get_cpus()
mini_buildd.util.list_get(list_, index, default=None)
mini_buildd.util.rmdirs(path)

Remove path recursively – succeed even if it does not exist in the first place

mini_buildd.util.json_pretty(json_obj)