ARCHITECTURE AND DATA MANAGEMENT IN THE AGE OF CLOUD, IOT AND MOBILITY
The popularity of composable architecture from 2020 and 2021 has continued in 2022 as well, especially on account of having independent, best-in-class, loosely coupled systems thus reducing cost, dependencies, time to market, and at the same time providing the best choice of services and personalised offerings to customers on an ongoing basis. Continuing focus on resilience and Business Continuity Planning (BCP) ensures this framework is scalable, replaceable, pluggable and enhance able through agile development. This is very important especially in the ongoing volatility, uncertainty, complexity and ambiguity (VUCA) world where organisations and architects need to be ever prepared for further black swan events and recovery, as this Deloitte research states.
Enterprise architects are focussed on leveraging more reusable components and best practices for quality assurance, storage, integration and other aspects with more involvement with key stakeholders while reducing technical debt and deciphering future business trends as this McKinsey article states. Composable architectures such as microservices based, API-first, cloud-native SaaS and headless (MACH) and packaged business capabilities (PBCs) are helping CIOs to easily, timely and cost effectively switch providers, adopt new technology stacks, and be more agile during transformation without dependencies on expensive consultants for these transformational exercises.
Gartner estimates that by 2023, organisations that have adopted a composable approach to ERP viz. best of breed solutions and API based approach would deploy new features 80% more quickly than their competitors. This approach also facilitates quick testing by sample populations and subsequent full-fledged implementations.
Besides the TOGAF and Zachman Framework for Enterprise Architecture, Data Management Capability Assessment Model (DCAM), Data Management Body of Knowledge (DAMA) -DMBOK 2 and other data architecture frameworks are increasing in popularity.
There has been immense acceleration in data generation velocity, volume and variety especially on account of the 2022 growth in the cloud, IoT and mobility ecosystem, along with edge computing and blockchain technologies. As organisations grapple with diversity in data sets across sensor, text, image, Audio-Visual, Voice, E-commerce, social media, besides the cloud, IoT and on-premise datasets, the role of data architects is assuming critical importance in terms of defining the relevance, models, quality & hygiene, policies, rules, and standards that govern the collection, storage, arrangement, integration, and use of these humongous data sets in its organisations across its lifecycle.
Data architects and their team are paying careful consideration in understanding, designing and having an end-to-end business and IT perspective of the data sources, metadata and schema, the data lifecycle pipeline of ingestion, cleaning, storage, analysis, delivery and visualisation, automation, APIs, cloud computing, container orchestration and storage, data streaming, AI/ ML models, analytics, and visualisation and security. They ensure data consistency and validation, implement ease to use interfaces, minimise data duplicity, movement and irrelevant versions, valid and comprehendible documentation, security and adherence to compliance, access and governance mechanisms and frameworks. Data architects have also been inculcating a data ops culture for ease and quick design, development and deployment of new components in the data architecture. Considering the VUCA world and expectation of near instantaneous latency in planning, reaction and response, it is of increasing importance The importance of maintaining, managing and leveraging observable data sets in an automated, orchestrated and integrated manner across the enterprise.
This article by McKinsey summarises the major considerations of data architecture which consist of Cloud with containerisation and serverless data, hybrid real time and batch data processing, shift from end to end commercially off the shelf (COTS) applications to modular best in function/ industry, move to APIs and decoupling, and from centralised data warehousing to domain based architecture and lastly from proprietary predefined datasets to data schema that is light and flexible, especially the NoSQL family.
It is also important to consider automated and active data management along with scaling, elasticity and decoupling hence incorporating independence of services, corresponding performance with relation to bursts and shutdowns, high availability, while optimising cost at the same time.
This Gartner article on technology trends for 2022 highlights that organisations have been looking at ML and API powered data fabrics along with data lakes, warehouses and layers to manage this data lifecycle by creating, maintaining and providing outputs to the consumers of this data.
Across BFSI, telecoms, healthcare and other industry verticals which are pursuing hybrid and industry cloud strategies along with containerised IoT platforms especially due to the need of customer data to reside locally, CXOs have been deploying hybrid data management environments, that leverage cloud data management tools to also automate, orchestrate, and re-use the on-premise and edge data, thus providing a unified data model and access interface to cloud, edge and on-premise datasets.
These rapid developments in the cloud and IoT ecosystems along with the rising FinOps ethos as mentioned in Part 1 of this series have necessitated infrastructure and data management to encompass dashboards, operational and control tools for end-to-end visibility, auto scaling and switching, and continuous optimisation of all the resources and usage across providers, business functions, and departments.
CIOs and enterprise/ data architects have been grappling with issues related to storage of this humongous data as well. This research by Statista estimated the total worldwide data amounting to over 27 Zettabytes in 2021, will exceed 180 Zettabytes in 2025. Leaders are deploying cloud, automation and orchestration, Storage-as-a Service (STaaS), decentralised blockchain powered data storage and storage on the edge. This ensures prioritisation of processes, tasks and resources to balance speed, efficiency, usage and cost along with eliminating security vulnerabilities and optimising backups and retrievals, and disaster recovery.
CDOs and CIOs have realised in 2022 that data literacy is not restricted to their offices and having a culture of data literacy along with business buy-in and alignment, open communication, robust processes and technology, top management focus and strong adherence to security, compliance and governance helps immensely. Considering the ongoing challenges in the workforce market such as quiet quitting, moonlighting, companies are focusing on having a readily available talent pool and also upskilling their pool of solution architects, data analysts and engineers.
BIG DATA, ARTIFICIAL INTELLIGENCE AND DECISIONING
2022 has seen the three interplaying factors of accelerated digital transformation especially with relation to widespread adoption of cloud, social media, IoT and mobility along with continuing challenges in maintaining customer, supply chain and employee experiences as well maintaining cost competitiveness, value-engineering, and responsiveness. This has made it even more critical for CIOs and CDOs to leverage their big, wide and small datasets from their internal and external functions, business processes and hardware and software touchpoints to enhance the total experience, be agile and responsive as well as maintain cost advantages and minimise downtimes.
Big data, analytics and artificial intelligence continues to be pivotal in managing customer and employee experiences through internet of behaviours, pre-emptive personalised experiences, launch, adoption, retention & referrals, upsell and cross sell recommendations. Data powered IoT, cloud, mobility and social media insights are immensely value-adding to consumer facing industries such as retail, healthcare, automotive, white goods, hospitality, entertainment, government and others. As mentioned in the previous part, data driven IoT is also instrumental in the Internet of Medical Things (IoMT), Internet of Packaging (IoP) and overall, citizen indices.
From the operations, maintenance, stores, logistics, and supply chain perspectives, data driven IoT along with other technologies such as additive manufacturing, robotics, digital twins, autonomic systems, AI, and smart metres are bringing about the benefits of operational, product and process excellence encompassing high quality, optimisation of costs, lower downtimes on predictive maintenance, improved inventory management, better logistics, equipment and staff surveillance, safety and tracking, employee motivation, and regulatory, environmental and emission compliance.
Let us examine this in the context of the automotive and healthcare industries.
This research by Mordor Intelligence predicts the big data market in the automotive vertical to reach close to USD 9 billion in 2026, with a CAGR of 16.81% during the period of 2021-2026. Automakers are applying AI and Analytics on their Big Datasets and in a multitude of use cases such as collating and analysing driving patterns, locations and trends, identification of part failures and root causes, increasing supply chain efficiencies and futureproofing against exigencies, better materials management, having targeted, efficient and cost competitive recalls, incorporating customer trends and needs and delivering a personalised, superior, economical and safe driving experience with better and reliable servicing maintenance. In the transport, trucks and buses category, AI, big data and analytics are being harnessed extensively for improving vehicle maintenance and service, enhancing safety, optimising routes and fuel efficiency, improving driver satisfaction and minimising fatigue as well as forecasting crew, spares, accessories, storage and other aspects in the ecosystem.
For passenger and electric vehicles, going one step further from the driving experience, owners and users are deriving superlative experiences from infotainment, location intelligence, charging infrastructure insights and new age car insurance products that incentivise better driving patterns along with vehicle health with lower premiums. Fleet management companies leverage telematics for better driver monitoring, identifying risks and improving visibility and control.
Thus, automotive companies have huge potential of additional revenues as well as savings from AI and big data in their IoT connected cars ecosystem. This McKinsey research predicts that by 2030, 95% of new vehicles sold shall be connected and this connectivity could deliver upto $310 in revenue and $180 in cost savings per vehicle yearly, on an average.
Deloitte research estimates the Internet of Medical Things (IoMT) market size to be valued at over $158 billion in 2022. Emergen Research predicts here that the global big data market in healthcare will exceed USD 78 billion by 2027 with a CAGR of slightly under 20%.
The data sources in healthcare: patients, healthcare companies, research companies, payers and other stakeholders have been churning data astronomically right from electronic health records (EHRs), imaging, genomic sequencing, smart sensors, mobiles and wearables, medical research, payer and patient information and portals, government records and many more.
AI and analytics are working on these big datasets and IoMT devices to monitor vitals, provide alerts and triggering workflows for better care management. Similarly, hospitals and pathology laboratories are enhancing their robotic surgery, imaging, telemedicine, emergency, trauma, ambulance, home healthcare and smart environment and sanitation functions. Thus, enabling players in the healthcare ecosystem including care givers, hospitals, digital apps providers, devices and wearables companies and healthcare technology solution providers to achieve better patient medical and emotional outcomes, proactive and preventive health management, critical disease handling, improve engagement and experience and reduce treatment time and costs.
Companies and healthcare providers are also aggregating these insights on populations and customers to draw Heatmaps, trends, proactive treatment plans, mapping hospitalisation risk levels and identifying gaps in facilities. With the focus on mental care and suicide prevention continuing throughout 2022, natural language processing (NLP), deep learning and analytics are being used on the big data wearables, social media, websites and internet of behaviour to identify patterns and draw our pre-emptive, preventive and proactive programmes.
Throughout 2022, research companies are also leveraging Big Data, AI, Deep Learning and Analytics in genomic sequencing, drug modules, drug discovery, testing and analysis, production and prediction of effects and precision medicine.
Even from a pure commercial function perspective, the increasing number of insurance claims frauds, health record breaches and cyberattacks in the healthcare vertical have also necessitated use of AI, big data and analytics along with cybersecurity tools and frameworks. Markets and Markets here estimates the healthcare fraud detection market to reach USD 5 billion by 2026. This ecosystem is also contributing immensely in boosting revenue, identifying pockets of leakage and optimise costs of payers, health systems, and life-sciences companies are increasingly seeking help from health services and technology (HST) firms to identify wasteful spending and boost revenue.
For achieving these outcomes in the customer facing, supply chain and operations end, data scientists and leaders in these and other industries have been amalgamating project management, statistical and planning tools, dashboards and user interfaces along these big datasets, data fabrics and AI/ ML in a structured, reusable and auditable decision intelligence framework. Adaptive AI is also improving decision intelligence systems by being more autonomic, self-learning and thus providing faster and better outcomes.
The global decision intelligence market is also in a growth mode, Report Linker here estimating it growing at a CAGR of 10%, and exceeding USD 15 billion by 2026.
As far as AI/ ML is considered, besides the current trends of intelligent RPA, hyperautomation, sentiment analysis, AI enabled virtual assistants/ chatbots and AI powered cybersecurity, in 2022 CIOs have been increasingly leveraging AIOps to enable faster automation, orchestration and deployment of AI in their enterprise wise production environments. SMEs have been making use of AI capabilities in the cloud and conversely, AI has immensely value-added to FinOps, orchestration, automation and continuous optimisation in the cloud. There is continuing focus on the 2020 and 2021 trends of generative AI and deep learning and as mentioned before, CIOs are also looking at adaptive AI to further enhance and make autonomic their decision intelligence systems.
The FIFA 2022 World Cup in Qatar has made extensive use of sensors, analytics and AI along with 5G connectivity, CCTV and security solutions to provide a host of benefits to the players, referees, spectators and organisers. Right from connected ball sensor and technology and AI powered offside and penalty area assistive technology with alerts, to spectator analytics and patterns/ algorithms for predicting events related to stadium security and disaster management.
As industry 5.0 takes off in 2023, it is expected that adoption of responsible AI enhances, thus enhancing trust, the human quotient and empowering people across the extended enterprise at the centre of all AI initiatives.
Incorporation of Zero Trust (ZT) and Cyber Resilience across inhouse or outsourced Security Operation Centres and Cybersecurity-as-a-Service (CSaaS) continue to surge throughout 2022, especially considering the proliferation of cloud, edge computing, IoT, and mobility, as well as reduced dependence on VPNs and continuing hybrid and remote working. This research by Deloitte estimates the worldwide Zero Trust Market to grow to almost USD 40 billion by 2024. Gartner’s paper here estimates that ZT Network Access is the fastest-growing segment in network security, rising 36% in 2022 and 31% in 2023, respectively. Furthermore, Gartner predicts that by 2025, at least 70% of new remote access deployments will be largely catered to by ZTNA vis-a-vis VPN services, up from less than 10% at the end of 2021.
There have been continuing cyber-attacks, breaches, leakages and attacks on OT/ critical infrastructures in 2022 including those on the Red Cross, T-Mobile, Toshiba, Meta, Oil Terminals in Belgium and Germany, energy companies in Italy and Greece, and various government facilities across Costa Rica, Latvia, Montenegro and Lithuania, besides the ongoing cyberwarfare between Russia and Ukraine. IT leadership especially in utilities, automotive, railway, airports, power plants, and pipelines hence give utmost importance to cybersecurity and resilience.
Gartner predicts that by 2025, 45% of organisations worldwide will have experienced attacks on their software supply chains, a three-fold increase from 2021.
From the mobility perspective, social engineering (phishing and smishing), trojans, distributed denial of service (DDoS), spoofing, malware, mobile ransomware, attacks on multiple layers across IoT devices and wearables, fraudulent Wi-Fi powered man in the middle (MiTM), and data leakage through malicious apps have beleaguered companies and mobile users throughout 2022 as well. In addition, targeted attacks on enterprise EMM, MDM and MAM systems have risen considerably.
Ransomware has continued affecting the manufacturing, government, automotive, transportation, construction, industrial services, technology, retail and healthcare verticals. Proliferation of “Big Game Hunter” ransomware gangs, vulnerabilities such as Log4j and PrintNightmare, Ransomware-as-a-Service (RaaS), the Russia Ukraine and other geopolitical crises are the main reasons. Ransomware has also caused an unprecedented national emergency in Costa Rica in 2022, with severe disruption of the finance, healthcare, social services and other government services, as well as private importers and exporters from this attack by the Conti group. The ongoing Russia – Ukraine conflict has also brought about increasing complexity in the ransomware ecosystem, especially considering the fact that nearly three-fourths of cryptocurrency-based ransomware payments have been traditionally directed to Russia.
This research by Garter towards the end of 2021, highlighted that ransomware and its new emerging models are a top threat to organisations. Ransomware attacks are now actively targeting the digital supply chain, as per this Gartner article of 2022. This recent paper by the World Economic Forum highlights the increase in malware and ransomware attacks by 358% and 435% respectively.
To pre-empt and alleviate these attacks and vulnerabilities, CIOs and CISOs continue to deploy ZT principles across all architectures, users, data, workloads and policies across on-premise, cloud and container environments, the edge, IoT devices, network devices, firewalls, users, endpoints and routers. These principles cover aspects such as Secure communication and micro segmentation-based traffic flow, data protection/ encryption/ anonymisation, least privilege user access and multi-factor authentication, DevSecOps and NoOps, automation and orchestration, and AI powered tools for external and internal threats detection, management and remediation, user behaviour analysis and insider risk management.
Growing adoption of industrial, consumer and commercial IoT and the gradual shift towards Industry 5.0, has necessitated CISOs to incorporate observability along with monitoring, of all assets, loads and health. Especially considering this complex, heterogenous ecosystem of applications, IoT devices, infrastructure, networks across cloud (multi/ hybrid and cloud native) as well as on-premise systems, it has been become important to have a unified platform to observe, monitor and have a clear field of vision across these diverse technology stacks, infrastructure and network traffic flow, to maintain high degrees of data, application health and cyber security posture management perspectives, thus delivering better and faster digital experiences, uptime, performance and strong security.
CISOs are hence integrating observability into the DevSecOps culture and lifecycle itself, thus making a discernible shift from the log and threshold alert-based monitoring system to a holistic optimal approach of root cause analysis and remediation of these complex hybrid OT environments and their constituent systems. Especially since observability encompasses Metrics, Events, Logs and Traces (MELT) and unlike logs and traces which can be turned off by the cyber intruders/ attackers, this network traffic cannot be turned off.
Besides ZT architecture, CIOs and CISOs are addressing these threats through Cyber Resilience across their IT and OT systems, supply chain, mobile and edge users, and extended frameworks with AI/ ML powered technologies such as anti-ransomware and ransomware protection/ threat detection systems, extended detection and response (XDR), security access service edge (SASE), identity and access management (IAM), cloud access security broker (CASB), cyber asset attack surface management (CAASM), security incident and environment management (SIEM), security orchestration and response (SOAR) tools and cyber data lakes.
Besides AI/ ML, enterprises are also leveraging other technologies such as blockchain in securing their edge device data, and RPA to create digital twins of the personal high security databases. They are also encompassing hardware encryption, zero trust software architecture, and cybersecurity tools and design to also cover supplier compliance and assessment exercises as well.
These tools are being augmented by infosec policies covering role-based access controls (RBAC) policies and multi factor authentication, ensuring updated OS and patch management, securing remote desktop protocols and active directory, regular security scanning, red teaming and penetration testing and identifying and addressing vulnerabilities such as plug-ins, and links.
Governments and private enterprises are embracing cyber resilience frameworks majorly based on Zero Trust Architecture such as The Cyber Resilience Review (CRR), Cybersecurity and Infrastructure Security Agency (CISA), Federal Information Security Management Act (FISMA), the National Institute of Standards and Technology (NIST) FIPS 199, 200 and 800-160 Volume 2 publications, the Homeland Security Act, Cybersecurity National Security Action Plan (CNAP) of the United States, European Union Agency for Cybersecurity (ENISA), the NIS Directive and the EU General Data Protection Regulation (GDPR).
Besides these frameworks, many governments are facilitating helplines, assistance, information resources, self-assessment tools and other guides for the public during steady state and during an adverse cyber event.
Private Enterprises are coming together along with Governments to address IoT Security concerns in this era of Zero Trust and Cyber Resilience. The ENISA in Europe, the NIST in the USA and the President’s May 2021 Executive Order are addressing cybersecurity concerns, guidelines and compliances for the IoT ecosystem. Private enterprises are working on crystallising IoT baseline security standards for consumer and industrial devices, shared security principles, driving basic security certifications, norms, and enforcing cooperation, transparency and conformance across supply chains and customers.
It is of paramount importance for CISOs and leaders to have an in-depth knowledge of country specific data privacy laws, especially for Multinational enterprises and those handling sensitive end customer and employee data. Aspects such as customer/ employee/ stakeholder consent and rights, data storage, retention and transmission policies, clear guidelines in case of infringement, and others must be carefully comprehended. Leaders must keep abreast of all developments across the world, especially across the states in the US, the AI Act, Digital Services and Market Acts of Europe, the new regulations across the Middle East, Japan, Thailand and so on and so forth
Since assets, users and entities are now across on-premise, at data centres, the edge and the cloud across the extended enterprise, decentralised risk and decision making, moving from Compliance and Security functions to Security Behaviour and Culture programs (SBCPs), consolidation and convergence of cyber security solutions and of vendors along with Cybersecurity Mesh Architecture (CSMA) help provide a proactive, uniform and integrated security framework and posture based on ZT.
Besides these architectures, tools, policies and frameworks, leadership teams have been working on having robust Backups, Recovery/ Restoration points, Disaster Recovery strategies and systems and Business Continuity Plans in place. Many organisations now have a Clear Ransomware Remediation Management Strategy covering all aspects right from the initial 3-4 days response, multiple payment scenarios to negotiations, recovery, switching to BCP modes, and incorporating regulatory frameworks, customer behaviour, legal contracts, negotiating powers and other factors. According to this McKinsey report, almost 2/3rd of the victim organisations actually pay the ransom despite the contrary usual pre-ransomware stance and large percentage of victims do not get their data back even after paying off the ransom.
Ransomware, Data Leakages, Breaches and its direct and indirect financial effects are being incorporated and accounted for within the Enterprise Risk Management (ERM) and organisational cyber insurance policies. The cyber insurance policies encompass 1st and 3rd part damages such as IT Forensics, Crisis Management Costs, Credit Protection, Crime and Social Engineering, Costs of Notification, Damages on account of Personally Identifiable Information (PII), breach of contract, Extortion, Social Media Damage Control costs, Ransomware and Social Engineering, damages related to viruses and negligent data protection, costs of interruption and restart, digital asset degradation and many other categories.
In 2022, many organisations have expanded ERM into an integrated Governance Risk and Compliance (GRC) framework also covering cyber risk, risk appetites, covers and tolerances. There are considerations of renaming the Chief Risk Officer to Chief Resilience Officer: and re-organising culture, processes, technologies, guidelines and workflows in consonance with Risk Appetite, KRIs and KPIs. This becomes very critical as there is a clear convergence of physical and cyber security which shall warrant robust Orchestration and Automated Response Systems.
This article by McKinsey suggests augmenting the more technical GRC to a more cross functional, business oriented cyber risk management information and reporting systems that provide leaders with the risk transparency they require for organisational resilience transformation. The cyber risk MIS is an integrated decision-support system, having visibility across all physical and cyber assets in the enterprise across Business Units and Regions and facility to define, detect, treat and measure cyber risk. Dashboards with risk heat maps provide the CISO and CRO with KRIs, KPIs, controls, and progress reports for different functions, organizational levels, and applications.
Lastly, IT and Security/ Compliance teams continue to work with CHROs to increase awareness and education amongst the employees, gig workers and contract staff on Insider Risks, Ransomware and other adverse Cyber Events, Enforcing Infosec policies covering best practices, Dos and Don’ts and checklists of Email, Browsing and Application access, along with escalation matrices and reporting mechanisms are the priority areas of enterprises. Considering the fact that cyber-awareness is still in a growing phase, companies are leveraging gamification and rewards and recognitions along with open communication, collaboration, and culture in the training and awareness campaigns.
Culture and Communication are hence of paramount importance and in 2022, leadership teams are relying upon building awareness and accountability of risk and security within business, running mock drills or crisis games to simulate the response during a mock cyber crisis, as this article by Deloitte mentions. Skilling is also extremely critical for the success of Cyber Resilience. As per this research by the World Economic Forum, 47% of surveyed companies have perceived shortcomings as far as their trained and skilled cyber security teams go. CHROs and CISOs/ CIOs are hence focusing on retention, upskilling and attracting the best talent.
промокод мелбет при регистрации. Click Here:👉 https://bit.ly/3YSgux0