Technological Requirements

Big Data has a lot of potential, however many technologies are needed. Here are some of the technological requirements we looked at in class:

Storage

To effectively use Big Data, there must be enough capacity to handle large data sets. This is because Big Data handles exabytes of information which, with current technology, requires a lot of physical space. Cloud storage is optimised for Big Data usage as it allows mass storage without a physical link. Certain applications are not ideal for Big Data as they lack the capacity required for large scale storage, traditional databases are a prime example of this. On the contrary, applications like Hadoop are tailored to handle large volumes of data. The surplus of generated data in modern times requires a constant need for additional storage, and more compact servers capable of holding more data.

Processing/Analysis

Once data has been compiled and stored, it can be managed and processed. This requires technology like data mining algorithms. Data mining can be conducted through various applications, many offered by Apache. The information is drawn from data lakes and data warehouses, where large scale data sets are stored. Data mining allows data points to be extracted for their value, giving insight to data analysts. Machine learning can also be utilised to process data automatically and efficiently. The physical hardware required to conduct larger scale data processing is quite advanced and costs a lot, however for small and medium scale data processing you can most likely use the device you're reading this on! How cool is that?

Visualisation

After data has been processed, it must be displayed. This can be done by putting data into graphs and charts. This can be done through a number of applications and websites easily as long as you can access the internet. Specialised applications can be used for larger data sets that still allow data to be displayed effectively. This allows conclusions to be drawn in a aesthetic, streamlined manner.

In summary, large data sets require special software and hardware which can be expensive for an average person, however this probably won't be an issue for companies and government institutions. Smaller sample sizes can still be tested by anyone with an average computer and access to the internet. 

References:

https://ilearn.fife.ac.uk/course/view.php?id=9751#section-17