The development of the ETCB information system started in the nineties. The systems still contain important parts that were developed back then and are based on the 20 year old technology. Over the years, the system has grown in an evolutionary way – there is more than one hundred different sub-systems supporting various business processes. Those systems have been built over the course of last 20 years, using different technologies, following different user interface guidelines and designs and different philosophies. The situation was complicated further by the merger of Estonian Tax Board and Estonian Customs Board in mid-2000 when their technologically disparate systems were also merged. The system was really heterogeneous in many ways, but at the same time contained big monolithic blocks of critical functionality that were implemented on the decades-old technology.
Around 2015, ETCB set an ambitious goal – to overhaul the whole system that was supporting the taxation in Estonia. Not just the information systems that were supporting the existing business processes, but also the business processes themselves. The expectations of the companies and citizens have changed – they expect much more openness, more proactive and streamlined communication and automated processes where manual work has been minimised.
The fragmented and outdated "face" of the ETCB systems – e-MTA or Electronic Tax and Customs Board – a self-service portal for companies and citizens where they can electronically perform all the tax and customs related activities – also needed a serious facelift and harmonisation. Nineties really wanted their UIs back.
So ETCB set up the e-MTA 2020 program with the goal to:
- streamline and modernise the tax collection business processes
- introduce a unified and modern user interface for all systems
- reduce the usage of legacy technology, to ensure the continuity of the ETCB operations
The program was kicked off in 2016 and first big results were demonstrated in September 2019 when the overhauled e-MTA went into production.
But before that, in May 2019 there was another launch that went more or less unnoticed, because the look and feel and functionality of the systems was not changed. The big changes were within the system.
During the last three years Cybernetica developed five new systems to modernise critical core functions of the ETCB system:
- authentication and session management
- access rights and delegations management
- persons data management
- user account management
- classifiers (or reference data) management
Those horizontal systems offer the core services that are needed by all other systems. In some cases (classifiers management, user account management) ETCB did not have a system in place to deal with those topics in system-wide manner. In other cases there were too many systems dealing with certain information (e.g. persons data management was scattered around several systems offering different services). Authentication and session management was still relying on proprietary solution developed in the early days of the e-MTA, more than 20 years ago. Access rights and delegations management needed harmonisation and better UI for companies and citizens.
But the last straw that broke the camel's back was the eIDAS regulation that mandated that all governmental institutions in all EU member-states should not discriminate the users of their electronic systems based on their citizenship. I.e. every EU citizen that has suitably secure electronic identity should be able to use all ETCB electronic services. And the system was not ready to handle this requirement. Lots of changes were needed in all systems, but the focus of the change was related to the management of persons, users, their access rights and delegations and authentication.
We took a broad view to the problem and designed a system for the future, a system capable of accommodating the future needs of ETCB. The key to the design was unification. We generalized the current requirements and came up with more general solutions that were needed to solve today's problems.
Probably one of the most important and perhaps harder to understand decisions was the separation of "persons" and "users". For us "user" is a technical concept - somebody that can use the system, somebody who has a user account in the system, the means to authenticate him/her/itself to the system and log into the system to perform some operations. It can be a human being, but it can also be an automated system. The term "person" on the other hand is a legal concept. In the context of ETCB, a person is somebody who is paying taxes. It can be a physical person or a legal person. It can be registered in Estonia, in some other EU member state or in some third country. It doesn't matter – in our systems all persons are treated in a same way. But a person never logs into a system. Log in is done by the user and in order to do something useful, they will select a person with representation rights; even if the selected person is the current user, access rights are still necessary.
The confusion that we encountered during the project stems from the fact that for many people the word "person" is synonymous to "a human being". For us it was a term that referred both to physical and legal persons – a generalisation. Most of the people who were known to the ETCB had two records – one in the user account management system and another in the person management system. But there were exceptions. Some people had only user accounts – e.g. when a person from other EU member state logs into ETCB system, a user account is automatically created for him/her. He/she can log in repeatedly and all the settings are stored. More importantly – after the user account is created, it is possible to assign access right to him/her to represent some person. So for example – Estonian company can hire a customs broker from Germany, who can log into ETCB system using his/her German authentication services and operate on behalf of the Estonian company as easily as Estonian people could do.
This separation between users and persons also helps to handle systematically important cases when human being is unfortunately not capable to represent him/herself.
So, all those complexities related to the peculiarities of particular human beings and organisations are handled by the small number of core systems and hidden from the hundreds of the systems that support the ETCB main business processes – we provide unified services to work with users, persons and access rights.
For every entity – whether it is a user or person we retain full history that contains all changes to the attributes together with the auditing information about when, why and by whom some attribute was changed. This allows us to build systems that work correctly across many years or decades, allowing precise correction of old tax and customs declarations – something that has always been a source of hard-to-deal errors.
There are lots of nice and small features built into the core systems that are not yet used by current ETCB systems but that offer great promise to the future.
"Sharing" and "friend-list" are features that make ETCB systems more flexible for traders and allow end-users to delegate their access rights to other users on the entity-to-entity basis. For example – the importer could share his/her access to customs declaration in the ETCB system to somebody from logistics company who needs this information to complete the cargo manifest. ETCB cannot foresee all the possible business relationships between companies and build this kind of access control logic into system. Traditional access controls are very inflexible and companies must resort to print-it-out-and-type-it-in type of information sharing approaches. "Sharing" helps to overcome this and puts the real data owners – the companies - back in control.
Another useful feature is "sub-user". Every user can create a sub-user and delegate some access rights to this sub-user. It becomes useful when the user gives the authentication credentials of the sub-user to some automated system, who can work on behalf of the user – but only using the limited set of services that the user delegated. You can imagine ERP system that can directly lodge some declarations using this lightweight mechanism. Or smartphone app that provides real-time view of the some information from ETCB system, without having powers to do something harmful, even when the phone is stolen.
It was real fun to design and implement a new standards-based authentication and session management system that supported all the legacy applications that ETCB had. We selected OpenID Connect as an underlying protocol for our authentication and session management system. On top of this we built a true SSO system that supports global sessions and single sign-out across ETCB applications – both the old ones that implemented UI on the server side using various technologies to the current crop of SPA applications written in JavaScript and invoking REST services provided by the back-end systems.
Building new functionality was just a part of the game. ETCB had decades worth of data that had to be migrated and made manageable by the new systems. The new system had to be compatible with old data models and business rules. We had to clean up the data during the migration process to compensate for the historical anomalies. Working with complex data models that have many complex business rules poses an interesting problem – how to make sure that the entities are correct and conform to all business rules. We decided to separate all business rules from Java code and use a rules engine for executing the rules. Historically we have written several business rules languages by ourselves, but this time we decided to use Drools. Rules are written and maintained by analysts. All entities are checked for consistency after every transaction – this way we make sure that we do not introduce erroneous data into the system. We also wrote small a helper application that fetched all entities (persons) from the database and validated them. We used this tool to validate the results of the migration and spot anomalous historical data.
We did many rehearsals of the migration Persons' data was migrated from Oracle to Postgres, converted to new data model, validated and tested more than thirty times. The results were analysed and migration scripts were fine-tuned after each run.
On the infrastructure side we moved away from Oracle DBMS and PL/SQL. New systems are written in Java and JavaScript, run on Tomcat application servers and use PostgreSQL databases. ETCB had to shut down their systems for five days in order to perform all the data migration and consolidation tasks, move the data from Oracle to PostgreSQL, update the majority of the systems (hundreds) to interface with the new systems. And while the system was already down, it was deemed to have good spot to upgrade the remaining Oracle databases to the latest version.
We would like to thank all the nice people from ETCB, RMIT (IT Centre of the Ministry of Finances) and other developers working on the e-MTA system for good cooperation.
Cybernetica has a long history of working with customs and tax systems. In fact the first Estonian tax declaration management system INFOTOLL was written already by the Institute of Cybernetics, back in 1991. Cybernetica supported and developed this system for nearly ten years.
In 2003 and 2004 we helped the Customs Board to upgrade the ASYCUDA++ system, developed it further to be compatible with EU customs procedures and systems and facilitated the EU accession of Estonia.
Starting from 2005 we have been continuously working with various large tax and customs systems. Today most of the ETCB customs business processes and several important tax business processes (including VAT and excise tax collection and management) are supported by systems developed by Cybernetica.
Written by Arne Ansper