Variant Namespaces Instead of MIME Type

[Florian Berger]

DE: Varianten: Namespace statt MIME-Typ

Varianten: statt MIME-Typ sowas wie Java/Android?-Namespaces: luna.html, ext.rakewi.quiz -> auch gleich API-Endpunkt-URL? /ext/

[florian]

• _"Namespaces are one honking great idea -- let's do more of those!"_
• The Java docs say on [Naming a Package](​https://docs.oracle.com/javase/tutorial/java/package/namingpkgs.html): _"Companies use their reversed Internet domain name to begin their package names"_
• The name should serve both _semantic_ and _accountability purposes_, i.e. we should be able to guess what it does, and how I can check it's legit.
  • [Dependency confusion](​https://www.activestate.com/resources/quick-reads/dependency-confusion/) is a thing.
    • For Apache Maven in the Java world, _"The group ID should resemble the package name, or reverse DNS of your web site"_ is a requirement to be listen in the [Maven Central repository](​https://repo.maven.apache.org/maven2/)
  • For Luna LMS that's easy: luna.text.html.ar_SY
  • But for 3rd parties, if we follow Java's style, how do we know where the domain ends? ext.org.example.user.audio.opus.fa_IR
    • The check would happen over some [well-known URI](​https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml), similar to DKIM. We could try from the most specific subdomain: stripping ext, then fa_IR.opus.audio.user.example.org -> opus.audio.user.example.org -> audio.user.example.org -> user.example.org -> example.org , until one of them succeeds.
• Finally, what about metadata? The variant type needs a readable name, probably a description etc.
  • A variant definition is part of the code, not of the data.
    • So: no definitions, code, metadata etc. on the _variant type and specification_ should lurk in the course database, in a local folder like icons etc.
    • Instead, it should be provided at the code level, in a folder where Python can load it.
      • That way, again similar to Java, the type name can become the path: import ext.org.example.user , which would imply a ext/org/example/user/__init__.py
    • This module can then register itself with a type, a localised name (or, in Dublin Core, an identifier), and a description

[florian]

The flow:

• HTTP GET request arrives
• Decide content type to retrieve:
  • If a specific mode is requested, use as content type
  • Else check Accept: header
  • Else fall back to text/html
• Try to retrieve content type from storage
  • If content type is not available, fall back to text/html

[florian]

Should we leave the MIME notation as is? I.e., use dots as delimiters, but not artificially split up fields?

• luna.text/html.ar_SY
• ext.org.example.user.audio/opus.fa_IR

This would also possibly give a hint as to where the domain might end.

[florian]

Language codes are a rabbit hole.

• ​https://cldr.unicode.org/index
  • ​https://www.unicode.org/cldr/charts/45/summary/root.html
• ​https://pypi.org/project/langcodes/
• ​https://pypi.org/project/PyICU/

[florian]

​https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/classicu_1_1Locale.html#a0638620f0b0c25aad55539f14f739244

>>> import icu
>>> icu.Locale.create
icu.Locale.createCanonical(        icu.Locale.createFromName(         icu.Locale.createKeywords(         icu.Locale.createUnicodeKeywords(
>>> icu.Locale.createFromName("de_DE")
<Locale: de_DE>
>>> str(icu.Locale.createFromName("de_DE"))
'de_DE'
>>> str(icu.Locale.createFromName("de"))
'de'
>>> icu.Locale.createCanonical("de_DE")
<Locale: de_DE>
>>> str(icu.Locale.createCanonical("de_DE"))
'de_DE'
>>> str(icu.Locale.createCanonical("de"))
'de'
>>> str(icu.Locale.createCanonical("fa_IR"))
'fa_IR'
>>> str(icu.Locale.createCanonical("fa-IR"))
'fa_IR'
>>> str(icu.Locale.createFromName("de"))
'de'
>>> str(icu.Locale.createFromName("de").minimizeSubtags())
'de'
>>> str(icu.Locale.createFromName("de_DE").minimizeSubtags())
'de'
>>> str(icu.Locale.createFromName("en-US").minimizeSubtags())
'en'

[florian]

So.

• Variants will in principle be described using RDF.
  • Minimum attributes are
    • dc:publisher as a DNS domain
    • ​MIME-Type
      • As on how to represent MIME in RDF, see the ​answer to "How to use MIME types for RDF typed literals"
    • dc:language with a minimised CLDR language
  • Variants may not be accessible by a dereferencable URI, so they will be referenced by their ID in a URN, such as urn:luna:variant:contentID:variantID
• From the attributes, a dot-delimited string is constructed by the above methods. The string serves as the format identifier.
  • It must be suitable for a Python import statement, i.e. obey the rules for ​Python package and module names.
  • The string may be used when requesting a mode from Luna, such as http://host/courses/view/courseID/contentID?mode=formatID
    • Since the mode string may be very long, verbose, and confusing to outsiders, a hash may be used instead. Currently the hash algorithm is SHA256. A prefix of the hash may be used as long as it is unique regarding the variants in the course.

[florian]

I briefly considered using the ​Accept-Header as the content format, but this is basically just MIME, and supports no further attributes. A language request would be a separate ​Accept-Language header, and that would become a mess pretty quickly.

[florian]

​https://rdflib.dev/ looks really useful to handle RDF in Python.

[florian]

contrib seems to be a more verbose prefix than ext.

See

• ​Django, ​git, ​OpenCV, ​PostgreSQL
• ​Answer to "What's in the "contrib" folder?"
• ​Add a "contrib" directory to your projects

[florian]

The Luna SQLite database does not store RDF.

This would lead to a possbily awkward duplication of data. Also, if semantic information would be kept in both columns and an RDF dump, how would Luna guarantee these are always in sync?

And if semantic information was exclusively kept in RDF records, then another query language and system would be needed on top of SQLite to retrieve the data. But a Luna storage backend should have a single API, not multiple.

But:

RDF should be easily retrievable from the database.

So probably a table is needed to map column names to URIs.

Consequences for variants:

• There will be a new table formats.
• Columns:
  • id – the dot-delimited string, foreign key for variants.format
  • publisher
  • media_type (MIME type)
  • language
  • sha256

[florian]

Implemented in ​d9dce7106a73a3eb3a7d0ae42fc99ac5dcf6dbd857fbbe3f3ae0ca633e4e0978 (Fossil) / ​14b8ff06 (GitLab?).