A data analyst will identify the various data sources an organisation might have with the intention of consolidating all the relevant data into a central repository. Interactive dashboards can then be built on this repository, giving a broad overview of the organisation’s pulse.
This process of finding these untapped sources and combining them into a single source is one of the most demanding steps in the data analysis process. After identifying the sources, data must be cleaned by removing any errors, filling in any missing data, and ensuring that data is consistent. During this cleaning process rules are generated to ensure that this work will be done automatically next time new data is loaded.
Having an up-to-date data repository fed by different departments of the organisation, one can start extracting knowledge from this data. The starting point is to have reports which show the overall direction of the organisation using interactive visuals. On top of this, the data analyst will use analytical tools to try to find patterns in data, identify anomalies that might have occurred, and even predict future possibilities.
The next trend in data analysis is to automate some of these tasks using machine learning techniques such as neural networks to generate insights, so that these patterns and anomalies are detected by these algorithms reducing the time required for a data analyst to manually comb through data.