IST Portal

Platformtools we use to create IST

Microsoft

IST has been fully developed using Microsoft development tools, on the .net technology and the IST metadata is on the MS SQL server. In statistical offices where IST is in full implementation all databases are on the MS SQL Server, although the IST does not require that.

MS SQL Server

We decided to use MS SQL Server because of its robustness. Many functions that others have to develop themselves, we simply leave to the MS SQL Server. Thus, the authentication and authorization are absolutely in the jurisdiction of Microsoft tools. The IST.net program uses Windows authentication. That is how it accesses the databases on the SQL server. It is not necessary to introduce rules for accessing some applications. The application set in the IST respects all roles, user accounts and their accesses to the tables of the appropriate database of the SQL server. Also, the whole reporting system is based on Microsoft tools.

Reporting and BI

The IST forwards the txt file with an SQL query or procedure (the path is located in the IST metadata) to the appropriate server, i.e. the database located on it. The SQL server operates as requested from the forwarded txt file, and if it is also requested, returns some of the data to the IST interpreter, which pairs it up with the appropriate header in Excel or XML format (the path to the header is also located in the IST metadata), and forms a report in Excel for the end user. Excel has set itself apart as a format suited for work reports, because of its ease of use and flexibility. The IST interpreter enables other output formats XML, JSON etc.

IST innovationhow do we do it

Issue

For any company, and IT within it, the problem arises if we approach a large number of applications in the traditional way, which is to make special applications for every research/project. It may then happen that applications are developed using different platforms (VB, C++, C#, Java...), that data is stored and processed in different places, from the server to the local machine, in various formats, from relational database to text files.

Dependence on various tehnical platforms and vendors is extrimely high and data chaos is in place. Business processes are slow and hard to control, burden on of human resources on redudant proceses grows and response to user needs is slowing down.

Data processing and process automation, not only in statistics, but anywhere, always consist of three main phases: data entry, data editing and finally data processing with report generation. Looking at things this way, we realize that we constantly write the same three programs.

Is it possible:

to have environment not dependent on constant technological changes?
to conduct statistical surveys with minimal efforts for programming... or even better, with no real programming at all?
to store applications in databases, besides storing data?
to be more resistant to IT staff turn over and reduce costs for IT licences?

Approach

We needed a development environment in which there is no programming, in which the applications are uniform, standardized, easy to make and not dependent on constant technological changes (new development tools, programming languages, platforms). We needed to have all data on same platform, even more, having it on same type of format or same type of relation database. It was obvious that going towards full data integration, from both the IT and statistical aspects, had been critical to ensure that data reached its maximum potential.

IST approach includes:

New data integration strategy
One platform for all data
One type of relational database for all data
One simple metadata with all relevant information about data
Unified approach to each app
Not dependent on technological changes

Solution

We decided that the basis of all should be what we know best - relational databases. Certainly the data are entered into the databases and reports are made against them. How about, besides storing data in databases to store applications in database, as well? We came up with the idea that it would be best to somehow describe the application in detail, to keep this description in a metadata database, and to develop a program that will interpret this knowledge base and in real time generate (based on descriptions from that database) and run data entry, data checking, data validation, data editing, advanced search with connecting data from different surveys, reporting and data processing.

So we came to the following concept:

A simple metadata database - IST metadata,
A metadata interpreter - IST interpreter

Apps	Describes application
Databases connection strings	Contains data (connection string, alias etc.) related to the database that is located in the background
Tables	Description of relational tables in which the data and the relations between them are located.
Columns	Describes the fields of the tables in the Tables. This is also where the rules, which the IST interpreter incorporates during data entry, are set.
Logical control and validation rules	Represents a set of batch logical control, automatic correction, data check and data validation rules related to statistic survey.
Reports and procedures	Definitions of reports and procedures. Each report or procedure is a combination of .XLS or .XML file and .SQL query or .SQL program.

IST Benefitswhy we use IST

For government

IST can be used by any government institution.

IST can be easily and efficiently implemented in other state institutions. In the SORS it is not only used for statistics, but also in the development of other necessary applications (all project management is handled using IST).

IST is an example of how in-sourcing can drastically reduce the amount of budgetary funds previously required for buying specialized software.

A single platform, a single program, a single metabase

Uniformity and an unique approach to data development can result in better communication between government offices

Simple education and application among IT experts

For the Institutions

Reduction of operating expenses, an increase of results.

A single platform, a single program, a single metabase

A unique approach to research and a unique technology for research processing that can also be used to process any other kinds of data.

Better coordination between statistical departments

Better and faster functioning of the statistical system

For data analysts

Simplifies the job of statistician

Basic knowledge of Windows environment

No need for training on how to use the application tools

Increased independence from the IT sector, which gives the IT sector more time to further develop the systems

Everyone with IST experience can immediately join a different project and replace a colleague.

For IT presonnel

More time can be dedicated to the project's methodology

A simple metabase (six tabels)

No further training required, just the knowledge of working with databases

Very easy on the programmers, can even be used without real programming (no need to write endless lines of code for data entry, control logic, tabling)

Transfer from support roles to roles that are as equally important as any other team member working on a project.

More energy and effort dedicated to testing new IT solutions (migration to the web, web services, cloud computing, semantic web, big data, etc.)

Cloud developmentwe develop IST in the Azure

We have chosen to develop in the Azure cloud. The Cloud proved to be the best solution for the development and maintenance of our business applications. The development of the IST interpreter is the first product of a public institution that has been transferred to this type of technology

ADJUSTABILITY - We can fully adjust the setting depending on our own needs. It is no longer a problem to set such configurations as are used in different statistics.

EFFICIENCY - We have increased the speed at which we work. The Cloud provides us with a possibility to use joint forces of differentstatistical offices in order to raise the IST to a higher level.

COST-EFFECTIVENESS - The Cloud allows us to have a drastic saving of budget funds since its use helps us eliminate costs related to procurement and maintenance of equipment that we need for development and testing of an application.

Have a question?

Contact IST team!