Big Data and Cloud Computing

There are many compelling reasons to move to consumption-based cloud computing; they include scalability, conversion of capital expenditures to operating expenses, and the ability to pay as you go.   Likewise, a similar pattern has been extended to human capital.

In a traditional data center, operators were often forced to choose between having excess capacity sitting idle or not being prepared for unexpected increases in demand.  Cloud computing is desirable because one of its characteristics is the ability to quickly increase capacity(Kilcioglu, Rao, Kannan, & McAfee, 2017; Microsoft, 2017).   Because the cloud computing suppliers have many customers with different demand patterns they can use shared hardware to absorb spikes in usage.  This is partly enabled through the use of virtualization technology. 

Data centers historically were part of an organization’s capital expenditures.  This implies considerable planning and justification for large expenditures.  Also, companies often were forced to deal with accounting procedures such as depreciation.  Consumption-based cloud computing allows for those costs to be treated as operational costs.  This also allows for tighter correlation of income and expenses. 

Related to this shift is the idea that data centers could shift from a “pay before you use it” to “pay as you go” model.  This allows organizations to launch offerings with lower risk.  The rationale being, if no one uses the service or offering the costs will be proportionately small.  Conversely, if an offering is exceptionally popular, the income will be sufficient to cover the related operational costs.

Cloud computing and the associated outsourcing of data center operations enables economies of scale.  Not only with hardware and infrastructure, but also operational elements.  If ten companies maintained their respective data centers, they would each need an incident management system, on-call support, and other employees tasked with keeping the center functioning.   Cloud computing enables, and encourages, a higher degree of automation and reduction in costs associated with operations.

This concept has been extended to other types of specialized skills.  For example, Data Scientists as a Service (DSaaS) is similar in that a firm can engage a scarce specialized ability as the need is encountered (Srinivasan & Vijayakumar, 2016).  Rather than attempting to recruit, retain, and engage a staff of highly trained analysts, a firm pays for them as they are needed.  In some ways, this can be compared to having a law firm on retainer instead of having in-house counsel.  Because data scientists often have specialized computation requirements, there is a natural synergy between cloud computing and DSaaS (Grossman, Heath, Murphy, Patterson, & Wells, 2016).

Cloud computing and associated consumption-based pricing have had a major impact on data centers.  Organizations are now forced to justify maintaining their data center, instead of relying on a cloud computing partner.  Most organizations do not need a dedicated data center.  Those few cases where they do will likely move towards a hybrid architecture, where much of the organization’s systems are in a public cloud.  To enable this approach, cloud computing companies like Microsoft are releasing tools to make managing hybrid environments easier (Microsoft, 2017).   The trend to outsource those areas and roles that are not key differentiators will likely continue, if not accelerate.

References

Grossman, R. L., Heath, A., Murphy, M., Patterson, M., & Wells, W. (2016). A case for data commons: Toward data science as a service. Computing in Science & Engineering, 18(5), 10-20.

Microsoft. (2017). Azure Stack – Azure On-premises.   Retrieved from https://azure.microsoft.com/en-us/overview/azure-stack/

Srinivasan, A., & Vijayakumar, V. (2016). Data Science as a Service on Cloud Platform. Paper presented at the Proceedings of the 3rd International Symposium on Big Data and Cloud Computing Challenges (ISBCC–16’).