Variant Namespaces Instead of MIME Type

[Florian Berger]

DE: Varianten: Namespace statt MIME-Typ

Varianten: statt MIME-Typ sowas wie Java/Android?-Namespaces: luna.html, ext.rakewi.quiz -> auch gleich API-Endpunkt-URL? /ext/

[florian]

​icu::Locale Class Reference: minimizeSubtags()

>>> import icu
>>> icu.Locale.create
icu.Locale.createCanonical(        icu.Locale.createFromName(         icu.Locale.createKeywords(         icu.Locale.createUnicodeKeywords(
>>> icu.Locale.createFromName("de_DE")
<Locale: de_DE>
>>> str(icu.Locale.createFromName("de_DE"))
'de_DE'
>>> str(icu.Locale.createFromName("de"))
'de'
>>> icu.Locale.createCanonical("de_DE")
<Locale: de_DE>
>>> str(icu.Locale.createCanonical("de_DE"))
'de_DE'
>>> str(icu.Locale.createCanonical("de"))
'de'
>>> str(icu.Locale.createCanonical("fa_IR"))
'fa_IR'
>>> str(icu.Locale.createCanonical("fa-IR"))
'fa_IR'
>>> str(icu.Locale.createFromName("de"))
'de'
>>> str(icu.Locale.createFromName("de").minimizeSubtags())
'de'
>>> str(icu.Locale.createFromName("de_DE").minimizeSubtags())
'de'
>>> str(icu.Locale.createFromName("en-US").minimizeSubtags())
'en'

[florian]

The flow:

• HTTP GET request arrives
• Decide content type to retrieve:
  • If a specific mode is requested, use as content type
  • Else check Accept: header
  • Else fall back to text/html
• Try to retrieve content type from storage
  • If content type is not available, fall back to text/html

[florian]

Should we leave the MIME notation as is? I.e., use dots as delimiters, but not artificially split up fields?

• luna.text/html.ar_SY
• ext.org.example.user.audio/opus.fa_IR

This would also possibly give a hint as to where the domain might end.

[florian]

Language codes are a rabbit hole.

• ​https://cldr.unicode.org/index
  • ​https://www.unicode.org/cldr/charts/45/summary/root.html
• ​https://pypi.org/project/langcodes/
• ​https://pypi.org/project/PyICU/

[florian]

​https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/classicu_1_1Locale.html#a0638620f0b0c25aad55539f14f739244

>>> import icu
>>> icu.Locale.create
icu.Locale.createCanonical(        icu.Locale.createFromName(         icu.Locale.createKeywords(         icu.Locale.createUnicodeKeywords(
>>> icu.Locale.createFromName("de_DE")
<Locale: de_DE>
>>> str(icu.Locale.createFromName("de_DE"))
'de_DE'
>>> str(icu.Locale.createFromName("de"))
'de'
>>> icu.Locale.createCanonical("de_DE")
<Locale: de_DE>
>>> str(icu.Locale.createCanonical("de_DE"))
'de_DE'
>>> str(icu.Locale.createCanonical("de"))
'de'
>>> str(icu.Locale.createCanonical("fa_IR"))
'fa_IR'
>>> str(icu.Locale.createCanonical("fa-IR"))
'fa_IR'
>>> str(icu.Locale.createFromName("de"))
'de'
>>> str(icu.Locale.createFromName("de").minimizeSubtags())
'de'
>>> str(icu.Locale.createFromName("de_DE").minimizeSubtags())
'de'
>>> str(icu.Locale.createFromName("en-US").minimizeSubtags())
'en'

[florian]

So.

• Variants will in principle be described using RDF.
  • Minimum attributes are
    • dc:publisher as a DNS domain
    • ​MIME-Type
      • As on how to represent MIME in RDF, see the ​answer to "How to use MIME types for RDF typed literals"
    • dc:language with a minimised CLDR language
  • Variants may not be accessible by a dereferencable URI, so they will be referenced by their ID in a URN, such as urn:luna:variant:contentID:variantID
• From the attributes, a dot-delimited string is constructed by the above methods. The string serves as the format identifier.
  • It must be suitable for a Python import statement, i.e. obey the rules for ​Python package and module names.
  • The string may be used when requesting a mode from Luna, such as http://host/courses/view/courseID/contentID?mode=formatID
    • Since the mode string may be very long, verbose, and confusing to outsiders, a hash may be used instead. Currently the hash algorithm is SHA256. A prefix of the hash may be used as long as it is unique regarding the variants in the course.

[florian]

I briefly considered using the ​Accept-Header as the content format, but this is basically just MIME, and supports no further attributes. A language request would be a separate ​Accept-Language header, and that would become a mess pretty quickly.

[florian]

​https://rdflib.dev/ looks really useful to handle RDF in Python.

[florian]

​contrib seems to be a more verbose prefix than ext.

See ​Django, ​git, ​OpenCV, ​PostgreSQL, ​Add a "contrib" directory to your projects

[florian]

The Luna SQLite database does not store RDF.

This would lead to a possbily awkward duplication of data. Also, if semantic information would be kept in both columns and an RDF dump, how would Luna guarantee these are always in sync?

And if semantic information was exclusively kept in RDF records, then another query language and system would be needed on top of SQLite to retrieve the data. But a Luna storage backend should have a single API, not multiple.

But:

RDF should be easily retrievable from the database.

So probably a table is needed to map column names to URIs.

Consequences for variants:

• There will be a new table formats.
• Columns:
  • id – the dot-delimited string, foreign key for variants.format
  • publisher
  • media_type (MIME type)
  • language
  • sha256

[florian]

Implemented in ​d9dce7106a73a3eb3a7d0ae42fc99ac5dcf6dbd857fbbe3f3ae0ca633e4e0978 (Fossil) / ​14b8ff06 (GitLab).