LargeInstanceProcessing
From XBRLWiki
(diff) ←Older revision | Current revision | Newer revision→ (diff)
Contents |
Introduction
Several families of taxonomies have led to potentially large instances (e.g. more than a few tens of kilobytes, up to several gigabytes).
The taxonomy currently known as having this characteristic are:
- Taxonomy of Bank of Indonesia, for which an XBRL White paper has been published (http://www.xbrl.org/sites/xbrl.org/files/imce/lrg_instance_proc_indonesia.pdf);
- Solvency II taxonomies defined by EIOPA (European Insurance and Occupational Pensions Authority: https://eiopa.europa.eu);
- Basel III / CRD IV taxonomies, COREP and FINREP, defined by EBA (European Banking Authority: http://www.eba.europa.eu).
Note: the European taxonomies are intended to be used by all countries of the European Union, and more.
The size of these instances are typically due to lists of details for things like loans, financial products or assets.
Some tests have been made and led to difficulties. The topic has been discussed in the XBRL International conferences of Abu Dhabi (reference needed????) and Yokohama (reference needed????). The subject is tackled by the XBRL International, in the Standards Board and Best Practices Board.
A Working Groupe Note has been published by XBRL International, proposing mainly to adopt a streaming solution and proposing adequate structure of XBRL instance.
This Wiki is a forum where this topic can be freely discussed.
Types of difficulties
Several difficulties may happen at different stages when processing instances, when:
- loading the taxonomy
- generating the instance
- signing the instance
- transmitting the instance
- parsing the instance
- validating the instance
- checking business rules
- reporting errors
- rendering the instance
Loading the taxonomy
In some case big instances correspond to big taxonomies.
When a Data Point Model appears in instances (case of highly dimensional taxonomies), instances are bigger than for moderately dimensional taxonomies, where some dimensional aspects are hidden. This large set of dimensional elements leads to big taxonomy.
Sometimes, it is necessary to chop a taxonomy in several entry points to avoid too big DTS, this is the case of the CORE taxonomy which had to be chopped in four parts.
In the case of multi-lingual taxonomies, like the European ones, existence of labels in several languages also inflate the size of the taxonomy. Care must be taken to include only used labels in a given country (there are 24 languages in the European Union, plus Norwegian and Icelandic).
Generating the instance
The FRIS document put constraints on the ordering of units and contexts that should appear before facts, but this rule must be relaxed because it hinders the streaming of the instances.
This aspect is covered by the Working Group Note.
Signing the instance
Typically, supervisors request the signing of the transmitted instances to fulfil integrity and non-repudiation.
Sometimes, it is also necessary to crypt the instance to fulfil confidentiality.
Security tool may have limitation and adequate tools must be used.
It could also be possible to sign or encrypt a compressed file but this would mean to have a canonical compression algorithm.
Transmitting the instance
Sending a multi-gigabyte document may cause difficulty but should be possible (technolgies exist to exchange video files of several gigabytes).
It is possible to transmit a compressed file that should be much smaller, due to the large compression factor of XML / XBRL files.
Parsing the instance
This aspect is covered by the Working Group Note.
Validating the instance
In this section, validation mean enforcement of the rules defined in XBRL 2.1 and XBRL Dimensions 1.0.
For the memory aspect, such a validation may be done fact by fact, with no need to keep the information in memory.
For dimensional validation, the context (or a representation of it) must be accessed, it is thus necessary to keep contexts-related information available.
