Data Technician Apprenticeship L3

 Big data - refers to a large or complex dataset that requires a different style of storage, processing and analysis techniques compared to more traditional data.

Semi-structured data - This is where the raw datum has additional information that defines the data structure within the data itself. Eg. XML; JOSN, HTML                 

Business Intelligence (BI) - BI (or sometimes MI - management information) is a term that describes the methods and tools used to use traditional structured data, from internal or reputable open sources looking to take an 'explanatory' approach to the analysis of the data.

Data Science - In contrast to BI, Data Science will take an 'exploratory' approach to analysis and will use a range of structured and unstructured data sets from multiple sources, open and public.

When considering data quality you need to consider :

Data source – Internal or External? from a trusted source? 
Data content – Is it current? Appropriate structure? Amount?

Some of the issues you may see with any data include:

-Data which has been altered (transformed) and some information lost e.g. rounding up of decimal places

-Merged data sets which will then contain duplications

-Data placed into inconsistent or across multiple columns and fields

-Data with no description or indication of relevance e.g. in a customer survey results data is 1 = Very good or Very Poor?

-Data coding that changes during the data set e.g. new product ID codes created and used for the same physical products

Descriptive analyics -  are used to collect, organise and present historical data to describe what has already happened. 

Diagnostic analytics - delves deeper into the description to try and answer 'why' did something happen.

Predictive analytics - are used to look into the future and what might happen based on past trends or patterns from the data available.

Prescriptive analytics - are the next step up from predictive analytics, where not only does it predict what could happen, but will consider multiple possible futures and aim to recommend the best course of action.