One company that is developing such risk intelligence is Prewave. Prewave is an AI spin-off that was created at the Vienna University of Technology through seed financing and investment from IST Cube. IST Cube is a new incubator and accelerator based at the Institute of Science and Technology in Austria, and Pioneer Ventures (TU Wien 2018). Prewave came about as a result of five years of intensive machine-learning research at the university (TU Wien 2018). Prewave’s technology aimed to develop an approach towards risk intelligence and management through the use of social media, news data, and supply-chain data and AI.
SIS technologies are advancing at a remarkable rate, and this may lead to many beneficial applications, such as predictive analytics. Harnessing this advancement, Prewave uses SIS to suggest predictive risk intelligence for improvements in areas such as supply chain management (SCM), insurance, and finance (WeXelerate 2018). However, the use of such technologies come with some ethical concerns such as data privacy, integrity, transparency and fairness, bias and the accuracy of predictive intelligence. Using Prewave as a case study, this paper addresses the research question: how do organisations working with predictive risk intelligence perceive ethical concerns related to SIS and in what ways do they deal with them? To address this research question, data were collected through a literature review, two semi-structured interviews with participants from Prewave, and a review of their website and the company’s case studies.
The paper presents a review of the current use of SIS in predictive risk intelligence. This is then followed by a discussion of a range of ethical issues surrounding the use and implementation of SIS in predictive risk intelligence. Further, the paper describes Prewave and outlines the objectives of using SIS in the organisation. It also discusses the effectiveness of SIS used in Prewave, before considering the impact of such systems on the organisation. Furthermore, the paper presents ethical issues that were identified in the use of SIS at Prewave. Following the identification of ethical issues, the paper presents remedial actions that are used to recognise and address ethical issues in the company. This paper contributes to the understanding of ethical issues when using SIS in predictive risk intelligence and supports the discourse that is aimed at addressing those issues.

The use of SIS in Predictive Risk Intelligence

The use of SIS in predictive risk intelligence is based on models that are developed using non-conventional approaches such as artificial neural networks and support vector regression (Kant & Sangwan, 2015). Such predictive analytical models can be used to identify both long- and short-term risks. Predictive risk intelligence is gaining ground because it leads to better allocation of resources, targeted prevention strategies, and improved decision support (Torous et al., 2018). To this effect, new statistical methods are being developed to optimally utilise existing data and make the most accurate predictions about risk (Kant & Sangwan, 2015; Torous et al., 2018).
Key to the use of SIS in predictive intelligence is machine learning. Machine learning is an AI approach to make predictions by learning from existing data instead of requiring additional programming (Cohen, Amarasingham, Shah, Xie, & Lo, 2014). Machine learning utilises modern computing and mathematical algorithms to build models based on available data sets, in which the model itself can improve with experience (Torous et al., 2018). Using machine learning, SIS are capable of recognising complex combinations of variables that reliably predict an outcome (Hall & Pesenti, 2017; Williams et al., 2018). SIS employs machine learning to analyse large, heterogeneous data sets that are then used to predict outcomes for a wide range of eventualities, including risk.  For instance, the modelling and predictive capabilities of SIS can be applied in industry and communities, which promotes efficiency and effectiveness of decision-making (Bentley et al., 2018).
The current generation of SIS is particularly suited for augmenting or automating tasks that involve at least some broadly defined predictive function. These cover a wide range of tasks, occupations and industries, from driving a car (predicting the correct direction to turn the steering wheel), diagnosing diseases (predicting its cause), to recommending a product (predicting what the customer will like), or writing a song (predicting which note sequence will be most popular) (Brynjolfsson, Rock, & Syverson, 2017).

Predictive Risk Intelligence in Supply Chain Management (SCM)

At the core of the use of SIS in predictive risk intelligence in supply chain management (SCM) is Big Data. The use of SIS carries with it the opportunity to change the SCM design and day-to-day decision-making. Thus, SIS are used to provide predictive intelligence through a range of techniques, resources, tools, and applications, ranging from baseline statistical analyses to advanced simulations (Waller & Fawcett, 2013). This growing combination of resources, tools, and applications has significant effects in the field of supply chain management. Some of the effects include improving forecasting accuracy, reducing costs and gaining better contextual intelligence across supply chain operations, which translate into lower costs and quicker response times to customers (Jeble et al., 2018; Waller & Fawcett, 2013).
SIS are also used to revolutionise supply chain dynamics through data sets, new methods of data science and new applications in the form of predictive analytics (Bendoly, 2016). Some of the methods that are used in SCM predictive analysis harness the potential of social media to provide large datasets (Singh, Shukla, & Mishra, 2018). Using social media in predictive analytics in SCM is prevalent. For instance, SIS are used for data capture at multiple points in the supply chain process in order to determine risks and opportunities. These data may be consumer sentiment data resulting from Tweets, Likes, and product reviews on websites (Singh et al., 2018; Waller & Fawcett, 2013).

Predictive Risk Intelligence in Insurance

Machine learning (ML) algorithms promise advancements in insurance risk management. Advances in computational power, the increasing amount of available risk data and the quality of data collected have helped SIS develop further when it comes to predicting risk and can, therefore, help in insurance decision-making processes (Baecke & Bocca, 2017). As is the case with other areas where SIS is used for predictive risk intelligence, the advances in SIS technologies allow ML algorithms to learn patterns in data which are indiscernible to human eyes and predict future risks (Jordan & Mitchell, 2015).  For instance, SIS based on these algorithms can automate an expert insurance analyst’s work, make routine decisions independently and raise problematic cases for review. As a result of such use of SIS in insurance, valuable expert time can be allocated where it is most needed while keeping the risk assessment up-to-date and available for all of its users. Automatic credit risk scoring systems, based on ML algorithms, are behind most current credit decisions and aid in calculating insurance premiums to cover different types of credit based on the risk of defaulting payment (Louzada, Ara, & Fernandes, 2016).

Predictive Risk Intelligence in Finance

In finance, SIS plays a significant role in predicting risks and future trends in the financial market, and in support of decision-making in trading financial instruments such as stock (Coyne, Madiraju, & Coelho, 2017; Fischer & Krauss, 2018; Geng, Bose, & Chen, 2015). Stock market prices are volatile. Wang et al. (2016) suggest that SIS can be used to make predictions in the stock market. It also serves as an early recommendation system for short-term investors and an early financial distress warning system for long-term shareholders. The predictions are made using price data to predict market index direction and stock price direction, and highlight risks and opportunities within the financial market (Fischer & Krauss, 2018; Geng et al., 2015). Therefore, SIS is also used by business owners, investors and policymakers alike to speculate on risks and trading trends (Coyne et al., 2017; Xia et al., 2013). For instance, investors and consumers use social media to share their thoughts and opinions, which creates a large amount of data that can be used for predictions. An example is StocKTwits, which is a social media platform used by investors to share information on stock trading (Coyne et al., 2017). Using SIS, such information is employed to understand and predict individual stock prices and determine risks associated with the stocks.

Predictive Risk Intelligence in Medicine

There is a growing use of SIS in predictive risk intelligence in medicine. For instance, rapid progress has been made in clinical analytics-techniques for analysing large quantities of data and gleaning new insights from that analysis to identify and manage high risk and high-cost patients (Bates, Saria, Ohno-Machado, Shah, & Escobar, 2014; Hamet &Tremblay, 2017). There are several opportunities to use SIS to reduce the costs of health care and to help identify high-risk patients, such as those at risk of cardiac arrest (Cohen et al., 2014; Petersen, 2018; Williams et al., 2018). Also, models based on ML can instantaneously consider the risk of all patients in a hospital and their individual therapeutic preferences (Cohen et al., 2014).

Predictive Risk Intelligence in Sustainability

Although progress with the use of SIS has largely been limited to areas such as medicine and finance (Keeso, 2015), there is a pressing need to integrate SIS within sustainability. Sustainability (environmental, social and commercial) is an emerging area for SIS applications. For example, some emphasise the need for SIS in the promotion of sustainability to attain business and strategic benefits (Hazen, Skipper, & Boone, 2016; Mani, Delgado, Hazen, & Patel, 2017). Hazen et al. (2016) and Mani et al. (2017) support the need for techniques that can process a large volume of data to gain actionable insights into environmental, social and economic sustainability.
Using SIS to generate insights at both strategic and operational levels is considered crucial for sustainability. Particularly in the supply chain’s operation-planning phase, SIS have been used widely to solve problems with procurement, inventory, and logistics (Hazen et al., 2016; Wang, Gunasekaran, Ngai, & Papadopoulos, 2016). Hazen et al. (2016) and Wang et al. (2016) further suggest that the use of SIS analytics can help predict and avert risks but also create innovative resources which can deliver strategic advantage and sustainability.
Correspondingly, a study by Chopra and Sodhi (2014) showed that the use of SIS in supply chain risk management could avert potential disruptions and help in rapid recovery from disruption. Such disruptions in the supply chain may be caused by various risks, including material flow risk, information flow risk and financial flow risk (Chopra & Sodhi, 2014).
In this section, several areas of the use of SIS in predictive risk intelligence were discussed to provide a picture of the state-of-the-art. Despite these positive uses of SIS in predictive risk intelligence, there are related ethical issues, which will be covered in the following section.

Ethical Issues with SIS in Predictive Risk Intelligence

The use of SIS in predictive risk intelligence raises several ethical issues. During background research into the ethical issues of using SIS in predictive risk intelligence, a number of sources were used. These sources included journals such as Science and the International Data Privacy Law journal. Also, insights were derived from proceedings from conferences such as International Conferences for Internet Technology and Secured Transactions to further understand the ethical issues with SIS technologies. In coming up with relevant literature, a broad search of keywords using different variations was conducted in databases such as Google Scholar and Scopus. After reviewing these articles, some ethical concerns with the use of SIS in predictive intelligence such as security and privacy, integrity, transparency and algorithmic bias were established (see Figure 1).

Security and Privacy Concerns

Ideally, predictive analytics or predictive intelligence involves linking data from multiple sources with social data to identify trends and provide insights for decision-making. With many new sources of data becoming available, such as data from social media applications, aggregating these data sources to achieve predictive analytics raises privacy concerns and requires new ways to preserve privacy (Bates et al., 2014; Petersen, 2018).
Several of the concerns, such as automated spear phishing and personalised propaganda, rely on the owners of SIS gaining unauthorised access to personal information about individuals. The risks posed by the use of SIS to security and privacy are exacerbated by poor threat-detection methods that misclassify malicious threats as benign, fail to detect key provocations or involve authentication mechanisms capable of misidentification and misinformation due to data misuse (Gupta, 2018; Eric Horvitz, 2017).
One example that highlighted some of the privacy concerns with SIS was the Cambridge Analytica-Facebook scandal. In 2016 Cambridge Analytica was involved in creating advantages for candidates in elections in the United States of America (USA) and the United Kingdom (UK). Cambridge Analytica, in conjunction with Facebook, was at the centre of harvesting and using personal data to influence the outcome of the US 2016 presidential election and the 2016 UK Brexit referendum. Cambridge Analytica used Big Data and advanced ML techniques to provide a full suite of services to enable highly targeted marketing and political campaigning, which raised concerns with regards to the privacy of those whose data had been accessed (Gupta, 2018; Isaak & Hanna, 2018).

Integrity

Another ethical issue with predictive risk intelligence relates to a lack of integrity when designing or using algorithms. For many companies, revealing certain information would have a knock-on effect on their business. Therefore, they may compromise the integrity of their processes in order to save their business (Hacker, 2018). There is a potential for an imbalance between a business’s interests and its moral obligations to other stakeholders. For instance, a company may propose a prediction that may improve particular social conditions around the world, but it may be unclear about its social obligations and to whom it is accountable. In a case where a company’s client uses morally unacceptable practices such as discriminatory profiling, there is a much higher likelihood of risk to the company which offers the predictive intelligence if this information is revealed, and so algorithms could be intentionally designed to ignore the practice (TU Wien, 2018).

Transparency and Fairness of Automated Decision-making

A further concern with the use of AI in predictive risk intelligence is the transparency and fairness of the algorithms used with the intelligence (Wachter, Mittelstadt, & Floridi, 2017b). According to Wachter et al., (2017), this concern arises because SIS use complex and opaque algorithmic mechanisms that can have many unintended and unexpected effects. When it comes to transparency and fairness in the automated decision-making process, such as predictive risk intelligence, users or clients only get a limited idea of why a decision has been made in a certain way, which does not mean the decision is justified or legitimate (Wachter, Mittelstadt, & Floridi, 2017a).
Some scholars, such as Hacker (2018), Horvitz and Mulligan, (2015) and Meira (2017), affirm that when it comes to the use of SIS in making decisions, for instance around risk, there may be a lack of transparency around what data is being used to train decision-making algorithms in the first place. A real-life example of such issues is the case of the Wonga payday lender in the United Kingdom. Wonga obscurely used more than 7,000 data points to assess how likely applicants were to default on a loan (Katwala, 2018).

Algorithmic Bias

There are also concerns about the reliability of using AI in making predictions. For example, AI can learn bias and prejudicial values when they are present within the dataset, leading to unfair or inaccurate predictions (Barocas & Selbst, 2016; Crawford & Calo, 2016). A lack of reliability in the predictions that are made by the SIS can be introduced when certain data are either included in or excluded from the training dataset (Williams et al., 2018). Due to potential bias in developing algorithms, AI can learn pre-existing inequalities that are present in the training dataset, resulting in a bias towards historically disadvantaged populations (Barocas & Selbst, 2016).
Additionally, data can be manipulated and misinterpreted according to the predispositions of those who are handling and manipulating data for making predictive intelligence (Katwala, 2018; Terzi, Terzi, & Sagiroglu, 2015).  An example of such bias is given by Hacker (2018) regarding the use of SIS in predictive medical intelligence. In predictive medical intelligence, the algorithm may reflect existing biases, with certain medical treatments being chosen on the basis of the practising physician’s speciality. Such issues highlight the importance of integrating data quality protocols and high ethical standards to mitigate bias and discrimination when using SIS for predictive intelligence (Hacker, 2018).

A Company using SIS for Predictive Risk Intelligence

One of the companies using SIS in predictive risk intelligence is Prewave. Prewave was incorporated in 2017 and stems from five years of advanced ML research conducted at the Vienna University of Technology. Prewave’s technology generates risk intelligence for its clients, derived from worldwide social media and news data. Using Prewave, a leading global logistics provider, has successfully integrated a working prototype for SCM applications, and it is hoped the technology will lead to substantial benefits for other markets, such as sustainability and insurance.

Prewave uses data retrieved from YouTube, Twitter, and local social networks and news media in each country being assessed to construct a risk analysis profile for companies and establish approximate predictions for their decision-making team (Dax, 2018). The SIS used in Prewave involves constantly learning and evolving AI techniques that can offer relevant, advanced predictions for a vast canvas of potential risks affecting business operations (TU Wien 2018). Prewave develops technologies to predict and detect social events using a variety of external data sources, including social media data.
The data collected were analysed using a thematic data analysis technique, which was both deductive and inductive. The deductive themes were derived from the interview questions to begin the analysis process. Using Nvivo qualitative data analysis software, categories were established into different nodes during a two-day SHERPA consortium workshop. Consequently, inductive themes emerged from the interview transcripts to inform this case study paper.

Individuals Interviewed

To obtain an in-depth understanding of the ethical issues related to the use of SIS in Prewave, two individuals from the company were interviewed after giving their consent to use their data for publication. The roles of the individuals interviewed included managing the operations of the start-up in controlling data science and business analysis, including marketing. Table 1 shows the interviewees and their roles.

 Table 1: Interviewee roles in Prewave

The SIS used by Prewave

The core technology used in Prewave is termed The Prewave Prediction Engine. The prediction engine uses smart information systems that enable the company to detect risk events on a global scale, days and sometimes even weeks before they happen (Prewave, 2018). Predictive intelligence is used to improve decision-making for various application domains and markets, ranging from Non-Governmental Organisations (NGOs) to supply chain management, corporate sustainability, and the insurance industry (Interviewee 1).
The SIS is used to analyse media streams with advanced data analytics technologies. This involves data retrieval from a range of social media sites, which is then analysed by ML algorithms to determine predictive outcomes for clients. The algorithms are trained in a number of languages and can evaluate a wide range of risk factors. Prewave’s primary background is the analysis of SCM and logistics, to determine possible supply chain disruptions and identify suitable alternatives, but its analytics have branched out to encompass the identification of insurance risks and business interruptions, as well as establishing issues affecting finance companies and sustainability agendas (WeXelerate, 2018). Prewave offers its predictive knowledge through plugins, application programming interfaces (API), and the Prewave web-based mapping (Prewave 2018).
Prewave used SIS to predict major national port workers strikes across seaports in Indonesia 18 days before they occurred (Prewave, 2017). The cooperative for workers responsible for loading and unloading cargo triggered the strikes because it was dissatisfied of its workers’ rights. Using SIS, Prewave was able to detect consistent pre-signals in social media messages and formulated an alert and risk level of impending strikes (Prewave, 2017).

The aims of Prewave in using these systems

There are several reasons Prewave uses SIS in support of its core business. The first interviewee mentioned that Prewave is a start-up that operates within the IT sector focussing on data analytics. Therefore, the SIS aims to analyse data and provide services to its clients on a range of decision-making processes. Interviewee 1 said:
‘we develop the algorithms, and the business model is to sell the generated data that enables our clients to take better-informed decisions. (Interviewee 1)
This was confirmed by Interviewee 2:
‘Prewave has the ability to extract insights that have the power to change practices in the world of enterprise, social, environmental, ethics and chain of custody intelligence to expose supply chain risk’. (Interviewee 2)
The SIS in Prewave is aimed at understanding real events affecting international supply chains, the discovery of critical elements across the impact value chain, and real-time knowledge for business continuity (Prewave, 2018a).
To encapsulate the aims of the SIS in Prewave, Table 2 shows the areas within the suppliers’ ecosystems where predictive intelligence offers virtual insights.

 
Table 2: Use of Predictive SIS in Suppliers’ Ecosystems (Adapted from www.Prewave.ai)
Prewave is currently piloting the use of systems from the insurance field to identify product risks, using SIS. An example was given by Interviewee 1 in relation to predicting risk for a cosmetics company:
‘if you have a cosmetics company producing shampoo or makeup, and if people are publishing that they get allergic reactions related to that product, that would be a risk that can show up on social media, for example, or in news data….So we focus on early detection, so many other companies do real-time risk monitoring, but we aim to detect the risks even before they happen […] very quickly in multiple languages’ (Interviewee 1).
In addition to using SIS in predicting a health and safety risk, Prewave also uses it in predictive finance by analysing data:
‘for investors to understand where their money is actually going and therefore make better decisions’ (Interviewee 1).
Similarly, the SIS is also aimed at providing local information on supply chains to help clients know if a supplier is risky to work with, by highlighting if there have been problems with that supplier in the past (Interviewee 1).
In terms of ethics, according to Interviewee 1, the SIS is aimed at highlighting:
‘wrongdoing by structuring unstructured information on a huge amount of issues and using that to blow a whistle in a non-aggressive way that can help the company, help the economy and help all the shareholders and stakeholders around the company’ (Interviewee 1).
To add to this, the SIS that is used in Prewave is aimed at:
‘making data structured and relevant for the right reasons to the right causes in a world with insane amounts of very messy unstructured data’ (Interviewee 2).

The Effectiveness of the SIS

The SIS used in Prewave is proving to be effective toward its aims and goals. This was suggested by Interviewee 2, who talked about the positive strides that the Prewave SIS is making. The interviewee said that the SIS:
‘has the ability to extract insights that have the power to change practices in the world of enterprise’. (Interviewee 2)
The SIS allows for better decision-making, which means a more informed understanding of what is entailed by certain decisions, the actions that will come from those decisions and any knock-on effects. Therefore, SIS has a big role in making decisions that are based on an informed understanding of the issues surrounding risk in insurance, supply chain management and sustainability.
According to the interviewees, the use of SIS in Prewave is so effective that the company is evaluating opportunities in different domains, such as finance.  The SIS has some good testimonials from users. It was indicated by Interviewee 1 that a client who invested in a company in a different country received a paper about potential risks there, and acknowledged that the information was crucial in understanding the problems faced by people living around the client’s company in that respective country.
Despite the effectiveness of the SIS thus far, Interviewee 1 admitted that there is still a need to have a human verification step within the process, considering that the SIS is still being developed. The interviewee further said that there is a need to improve and learn how to calibrate the AI better.
Regarding the maturity of the SIS in Prewave, according to Interviewee 2, the system is in its growth stage, and there is still room for it to mature and be refined so that the processes become more effective and more efficient over time. For example, the website is still being developed to make it more user-friendly and informative to increase online traffic.

General Impacts of the SIS

The SIS has some positive impacts on society and industry. The target market of the SIS is not localised to a geographical region but has also shown impact in several parts of the world (Prewave, 2018b; TU Wien, 2018). One of the impacts of the SIS has been its capability to detect positive use cases that are used to limit risk in areas that affect local and international economies such as finance and sustainability.
For instance, Prewave is trying to launch a pilot with labour unions in different countries to see if they could use the SIS to predict strikes and help the unions use the latest technologies to know what is going on, and support their members effectively. Interviewee 1 gave an example whereby Prewave provided predictive intelligence on labour unrest and industrial accidents to a Non-Governmental Organisation in Hong Kong. The predictive intelligence provided information that was published in an effort to make the labour unrests and industrial accidents transparent.

Stakeholders of Prewave SIS

The stakeholders of Prewave’s SIS are wide-ranging. Interviewee 2 said that the stakeholders include:
‘mother earth, the employees, the communities around the world that we’d extract data from, the public and the investors behind us’ (Interviewee 2).
This shows that potential stakeholders for SIS range from individuals (citizens) to governments. For example, labour unions are interested, along with people from different organisations, NGOs, the investor community, and the general public.
So far, this paper has covered the use of SIS in Prewave and its effectiveness towards achieving the organisation’s aims and the aims of the different stakeholders. However, despite all the positive impact that predictive intelligence could offer, there are some ethical issues. A number of them have been highlighted in the literature as briefly discussed in section 3 above. The next section outlines the ethical issues that emerged specifically with the use of SIS in Prewave.

Ethical issues with the SIS used in Prewave

A number of ethical issues arose from the use of SIS for predictive risk intelligence. Some of the ethical issues that emerged correspond to those that have been covered in the literature with regards to the use of SIS in predictive intelligence in supply chain management, insurance, finance, medicine and sustainability.  The ethical issues that emerged from the two interviews related to privacy and data protection, transparency and accountability, reliability, and finally trust and accuracy of the predictive intelligence (see Figure 3). These ethical issues are discussed below.

Privacy and Data Protection

The first ethical concern that came out of the interviews was to do with privacy and data protection. The use of social media data has many implications for privacy, as has been seen in the case of Cambridge Analytica, which was using Facebook data for predictive analysis (Isaak and Hanna, 2018). With the use of social media data, there is the likelihood that the SIS could have unauthorised access to the personal data of unsuspecting social media users whose data are collected for predictions, which poses a risk to privacy. Considering that the SIS used for generating the predictive intelligence collects a lot of social media data, this may pose a concern, especially if there are outsiders such as hackers who could get hold of the data collected and use it for purposes that were otherwise unintended.  However, although there is such concern, in the case of Prewave this in not really a serious concern because the company uses social media data and public news data that can be easily collected by anyone with programming skills.
The issue of privacy and data protection is also linked to the malleable nature of the techniques used to predict risk intelligence. Despite having security procedures in place, the techniques used in predictive intelligence could have disruptive effects in the flow of ideas, and access to information that results from the advancement of innovation around the information marketplace. This risk is further exacerbated by the interconnectivity of technologies that are used in SIS whereby localised systems are becoming more integrated into larger systems that govern every aspect of people’s lives such as social media. Such larger systems are being controlled by multi-national organisations or governments which has a lot of implications on the privacy of the data being collected and stored.

Transparency and Accountability

From the interviews, the second ethical issue relating to the use of SIS in Prewave is transparency and accountability. It was established that the processes used to cultivate predictive intelligence are not yet transparent. For instance, Interviewee 1 mentioned:
‘[people] understand what we are doing, but they do not know how exactly we are doing it; and we actually need to keep it secret in order to protect our intellectual property’ as requesting a patent is difficult, cost and time intensive in the software industry. (Interviewee 1)
On one hand the issue of transparency, in this case, seems to be a result of protecting the business, which could be argued to be valid for Prewave’s survival. However, it is still an ethical concern when there is no transparency in how the data is collected and manipulated. This was also confirmed by Interviewee 2 who said:
‘we actually try to keep it secret […] which is sometimes a bit of a problem if you want to talk to universities and collaborate’. (Interviewee 2)
On the other hand, to show how the issue of accountability is significant in the use of SIS in Prewave, Interviewee 1 stated that:
‘Twitter actually asked us how happy we are with the services they are providing, they are generally also interested into innovative usages of their data and services, but more in terms of understanding the value they deliver to their clients less in terms of compliance and transparency’
This shows that the issue of accountability is relevant on both sides of the process, for the data subjects (the public), and the data providers.
The issue of transparency in using the SIS to predict risk intelligence also has implications for informed consent. The assumption is that since the data is public, the data subjects do not explicitly object to others using it. This was suggested by Interviewee 1 who said that:
‘By using a platform, users agree or even ask a platform to publish their posts. That means when users publicly post information these data are considered to be public and can therefore be read by any party, once the data is public, users do not know who analyzes or reads these data. (Interviewee 1).
However, with regards to internal transparency, Prewavepapers its activities to its funders and its employees through regular meetings. For example, Interviewee 1 stated that public funding of organisations in Austria triggered a lot of discussion around ethics and human rights, because as a start-up Prewave uses both private equity and public funding, and so there is a need for accountability and transparency to these public organisations where the risks and benefits of the technology are discussed.

Bias

The third ethical consideration with the use of SIS in Prewave is bias. The issue with bias in the use of SIS can manifest through the selection of clients for predictive risk analysis. Interviewee 1 stated that the company ‘consciously’ selects its clients. This sounds as if there are some preconditions in selecting clients who would work well with the Prewave system. In this case, there is a high chance of bias or predisposition which could have implications on how the algorithms are designed and used.
For example, Interviewee 1 mentioned that the sample to use for predictive intelligence is preconceived to give certain expected outcomes. The interviewee said that there are objectives when picking the sample size so that samples of potential interest are selected, which could give certain results, as mentioned in the excerpt below:
‘… if there is an abnormality, then we would detect it as something that is potentially wrong or potentially of interest, like a corrupt event. If the signal comes from several authors, then the signal is more significant and we can pick it up’.

Trust and Accuracy

The fourth ethical issue is around trust and the accuracy of the SIS in predicting risk.  This may be a result of using misrepresentative data or misrepresenting information, as suggested by Interviewee 1:
‘if we paint a different picture to what it actually happens in reality, only because the public representation looks like that […], that’s a real issue, and that’s a risk that could arise’ just because of the fact that we rely on social and news media data. (Interviewee 1)
The interviewee acknowledged that there is a risk with the accuracy of the predictive intelligence. If the intelligence is inaccurate, it will have a knock-on effect on the trust of the clients or stakeholders that use such information. Interviewee 1 added that the concern is that:
‘predictions can only be as good as the underlying data. If the underlying data is biased, predictions will also be biased. Potentially biases arise from regions where social media is not used (or allowed), or where “Fake news” or propaganda information is intentionally spread’
When it comes to using Big Data, Interviewee 2 indicated that accuracy is fundamental because of the influences that surround the data manipulation such as ‘data promiscuity’.  Data promiscuity is the desire to mix data for indiscriminate purposes and so has the potential to lead to inaccurate outcomes.  Interviewee 2 also stated that with the use of SIS in predicting risk, it is important that the:
‘outcomes are more targeted and are a clearer representation of that data that can be used for some sort of betterment’. (Interviewee 2)
The interviewee further said that an accurate identification or a prediction is something that can be used for a good decision, rather than going towards unclear outcomes, therefore:
‘[the] problem is finding a clear way […] to use the data in the best way possible. Have it measurable, visible and tangible in front of you as opposed to just getting caught up in the flood of buzzwords’ (Interviewee 2),
The issue of accuracy is also connected to the effectiveness of the SIS when predicting risk. Prewave acknowledges the limitation of the SIS that is used in the company and therefore considers involving human intervention in ensuring accuracy. Interviewee 2 said:
‘we need to find out what our clients would tolerate. If they say, okay, they want 99% accuracy and no wrong alerts at all […] then I think we will always have the last step where a human, at least for some of the cases where our machine says “I’m not sure,” needs to make the last decision. So I think it will be hard to eliminate the human totally’. (Interviewee 2)
In this section, we covered some ethical issues that are related to the use of SIS in Prewave. It is fair to say that the company recognises most of the issues that emerged in the literature review such as privacy, integrity, transparency and bias. To show such recognition, Prewave has put in place some measures to address ethical issues. These remedial measures are presented in the next section.

Prewave’s Effort to Address the Ethical Issues

Prewaverecognises that it functions in a time of advancing technology when data privacy implications are prevalent. In order to ensure that the issue of data privacy is addressed in a way that mitigates the risks connected with data breaches and lawsuits, the conducts responsible innovation by following Responsible Data Science (see Figure 4 below).

Responsible Data Science involves methods aimed at limiting the potential for misuse of personal data and the risk of undermining public trust through fairness, accuracy, confidentiality, and transparency of data use (van der Aalst, Bichler, & Heinzl, 2017). Under Responsible Data Science, Prewave aims to adhere to a code of ethics. The code of ethics is aimed at ensuring that the use of SIS in Prewave remains legal, ethical, socially responsible and accountable to different stakeholders (Prewave, 2018c). As pointed out in section 5.3 above, transparency is one of the ethical issues that relate to the use of SIS in Prewave’s predictive intelligence business. However, Prewave is aware of the ethical concern and has put provisions within its code of ethics to address it.
Prewave also has a data privacy and protection plan which covers the use of lawfully obtained publicly available or proprietary data; aggregation and anonymisation of meta-data of users, and adherence to all applicable data protection laws (Prewave, 2018b). Further, the company’s data protection approach includes a commitment not to grant full exclusivity on generated social unrest event data to any single party, unless the data is used to minimise the risk of suppression of unrest events or to protect the violation of human rights (Prewave, 2018b).
Under Responsible Data Science, Prewave has access to an Ethics Committee, which provides advice on ethical issues related to the research and work that is carried out in the company. In addition to the Ethics Committee, Prewave engages a variety of stakeholders in its operations (see Figure 5):
‘through dialogue, posts, case studies […] that would highlight parts of Prewave development’. (Interviewee 2)
This includes:
‘meeting with external stakeholders to ask for their opinions, and to find out about the risks and develop ways to tackle them’. (Interviewee 2)

From the discussion above, it can be learnt that although the use of SIS in Prewave poses some ethical concerns, there are remedial actions that have been put in place to address the issues raised in this case study.  Similar companies that use SIS for predictive intelligence could learn from the remedial measures that are taken by Prewave and apply them in their business.

Conclusion

The Prewave case study has highlighted the use of Smart Information Systems (SIS) in predictive risk intelligence in domains such as insurance, supply chain management and sustainability, in addition to medicine and health science where it has predominantly been applied.  In addition to the benefits of using SIS to predict risk intelligence used to inform some decision-making processes for better outcomes in industry or communities, the case study also highlighted some related ethical issues. The issues identified from the interviews in section 5 are comparable to those discussed in the literature in Section 3 and shows that the use of SIS in predictive risk intelligence poses similar ethical issues regardless of the sector in which the technology is used.  Therefore, the main ethical issues that have emerged from the case study include protection of the data being used in predicting risk, data privacy of the data subjects and consent from those whose data has been collected from data providers such as social media sites. Also, there are issues relating to the transparency and accountability of processes used in predictive intelligence. Further, the interviews highlighted the possibility of bias in using the SIS for making predictions for specific target clients and therefore formulating algorithms that will service those clients. The last ethical issue was related to trust and accuracy of the predictions of the SIS.
Prewaverecognises the possible ethical concerns that relate to the use of SIS in their operations and therefore have put some remedial measures in place to address these issues. For instance, there is an ethical code of conduct together with the active engagement of stakeholders in discussing some of the activities that the company is carrying out.

Limitations

While a case can be made for the ethical issues identified in this paper, it is arguable that the most impressive capabilities of the SIS and therefore the ethical issues related to its use in predictive risk intelligence have not yet been widely explored. Considering that Prewave is a start-up and the technology is still being developed and going through its growth stage, more could emerge over time regarding the ethical implications of the technology. The issues identified in this study are a starting point and only reveal that there are ethical implications for companies using similar SIS to consider. The size of the company and its stage in the development cycle could not give a broader picture because like other technologies, the full ethical effect of SIS use in predictive risk intelligence in SCM, insurance and sustainability will not be realised until waves of complementary innovations are developed and implemented.

Contribution

There is extensive research on predictive intelligence in medical sciences, but there is a need to develop knowledge in other fields. The case study on Prewave has offered a significant contribution to the discourse on the implementation of SIS, and also around the ethics of AI by highlighting some of the ethical issues that result from the many uses of SIS.  Therefore, the case study has offered a new perception on the use of SIS and its ethical implications in predictive risk intelligence.

Implications of the Case Study

The case study has implications for theory since it presents original insights from a company that uses SIS in predictive intelligence in SCM, insurance and sustainability. The paper has practical implications for the implementation of SIS in industry, particularly for start-ups such as Prewave. Ethics should be a central consideration for companies and individuals developing SIS, in order to create meaningful positive change for society. The ethical issues resulting from predictive risk intelligence in the areas in which the company operates need to be identified and discussed so that they can be addressed. In so doing, companies including start-ups, which are developing similar SIS could use the case study to reflect on their practices and the ethical implications towards society in the future. For instance, the case study has practical implications relating to responsible and inclusive delivery of warnings for events. It has never been more imperative to have an open discussion about the proliferation of technology and how it will affect privacy rights and data security on both personal and national levels. This case study supports such a discussion by showcasing how it is also imperative for researchers, and innovators to take heed of the ethical issues and continue pondering on remedial actions.

Further research

While the Prewave case study provides a preview of the ethical issues with predictive risk intelligence, there is a need to get an understanding of the larger picture of the issues that are significant when it comes to using SIS in predictive risk intelligence. This should not stop at risk, but also address other aspects of society such as societal welfare. Further research is required to uncover additional ethical issues that relate to this emergent field, with a particular interest in evaluating and identifying the best ways of addressing the ethical issues. Thus, the research would benefit from additional case studies from bigger and leading companies in the sector. It would also be interesting to evaluate the differences in how ethical issues are recognised and addressed between smaller and well-established bigger companies to ascertain mutual learning mechanisms within the sector.

References

Baecke, P., & Bocca, L. (2017). The Value of Vehicle Telematics Data in Insurance Risk Selection Processes. Decis. Support Syst., 98(C), 69–79. https://doi.org/10.1016/j.dss.2017.04.009
Barocas, S., & Selbst, A. D. (2016). Big Data’s Disparate Impact. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2477899
Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014). Big Data In Health Care: Using Analytics To Identify And Manage High-Risk And High-Cost Patients. Health Affairs, 33(7), 1123–1131. https://doi.org/10.1377/hlthaff.2014.0041
Bendoly, E. (2016). Fit, Bias, and Enacted Sensemaking in Data Visualization: Frameworks for Continuous Development in Operations and Supply Chain Management Analytics. Journal of Business Logistics, 37(1), 6–17. https://doi.org/10.1111/jbl.12113
Bentley, P. J., Brundage, M., Häggström, O., Metzinger, T., European Parliament, European Parliamentary Research Service, & Scientific Foresight Unit. (2018). Should we fear artificial intelligence?: in-depth analysis. Retrieved from http://www.europarl.europa.eu/RegData/etudes/IDAN/2018/614547/EPRS_IDA(2018)614547_EN.pdf
Brynjolfsson, E., Rock, D., & Syverson, C. (2017). Artificial Intelligence and the Modern Productivity Paradox: A Clash of Expectations and Statistics (No. w24001). https://doi.org/10.3386/w24001
Chopra, S., & Sodhi, M. S. (2014). Reducing the risk of supply chain disruptions. MIT Sloan Management Review, 55(3), 73–80.
Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014). The Legal And Ethical Concerns That Arise From Using Complex Predictive Analytics In Health Care. Health Affairs, 33(7), 1139–1147. https://doi.org/10.1377/hlthaff.2014.0048
Coyne, S., Madiraju, P., & Coelho, J. (2017). Forecasting Stock Prices Using Social Media Analysis. 2017 IEEE 15th Intl Conf on Dependable, Autonomic and Secure Computing, 15th Intl Conf on Pervasive Intelligence and Computing, 3rd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), 1031–1038. https://doi.org/10.1109/DASC-PICom-DataCom-CyberSciTec.2017.169
Crawford, K., & Calo, R. (2016). There is a blind spot in AI research. Nature, 538(7625), 311–313. https://doi.org/10.1038/538311a
Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654–669. https://doi.org/10.1016/j.ejor.2017.11.054
Geng, R., Bose, I., & Chen, X. (2015). Prediction of financial distress: An empirical study of listed Chinese companies using data mining. European Journal of Operational Research, 241(1), 236–247. https://doi.org/10.1016/j.ejor.2014.08.016
Gupta, A. (2018). THE EVOLUTION OF FRAUD: ETHICAL IMPLICATIONS IN THE AGE OF LARGE-SCALE DATA BREACHES AND WIDESPREAD ARTIFICIAL INTELLIGENCE SOLUTIONS DEPLOYMENT. (1), 7.
Hacker, P. (2018). Teaching Fairness to Artificial Intelligence: Existing and Novel Strategies Against Algorithmic Discrimination Under EU Law (SSRN Scholarly Paper No. ID 3164973). Retrieved from Social Science Research Network website: https://papers.ssrn.com/abstract=3164973
Hall, W., & Pesenti, J. (2017). Growing the Artificial Intelligence Industry in the UK. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/652097/Growing_the_artificial_intelligence_industry_in_the_UK.pdf
Hamet, P., & Tremblay, J. (2017). Artificial intelligence in medicine. Metabolism – Clinical and Experimental, 69, S36–S40. https://doi.org/10.1016/j.metabol.2017.01.011
Hazen, B. T., Skipper, J. B., & Boone, C. A. (2016). Big data and predictive analytics for supply chain sustainability: A theory-driven research agenda. Computers & Industrial Engineering, 101, 592–598.
Horvitz, E, & Mulligan, D. (2015). Data, privacy, and the greater good. Science, 349(6245), 253–255.
Horvitz, Eric. (2017). AI, people, and society. Science, 357(6346), 7–7. https://doi.org/10.1126/science.aao2466
Isaak, J., & Hanna, M. J. (2018). User Data Privacy: Facebook, Cambridge Analytica, and Privacy Protection. Computer, 51(8), 56–59. https://doi.org/10.1109/MC.2018.3191268
Jeble, S., Dubey, R., Childe, S. J., Papadopoulos, T., Roubaud, D., & Prakash, A. (2018). Impact of big data and predictive analytics capability on supply chain sustainability. The International Journal of Logistics Management, 29(2), 513–538. https://doi.org/10.1108/IJLM-05-2017-0134
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415
Kant, G., & Sangwan, K. S. (2015). Predictive Modeling for Power Consumption in Machining Using Artificial Intelligence Techniques. Procedia CIRP, 26, 403–407. https://doi.org/10.1016/j.procir.2014.07.072
Katwala, A. (2018, December 11). How to make algorithms fair when you don’t know what they’re doing. Wired UK. Retrieved from https://www.wired.co.uk/article/ai-bias-black-box-sandra-wachter
Keeso, A. (2015, September 15). Big Data and Environmental Sustainability: A Conversation Starter (in brief). Retrieved December 17, 2018, from Medium website: https://medium.com/@AlanKeeso/big-data-and-environmental-sustainability-a-conversation-starter-in-brief-4052d0b2f0ae
Louzada, F., Ara, A., & Fernandes, G. B. (2016). Classification methods applied to credit scoring: Systematic review and overall comparison. Surveys in Operations Research and Management Science, 21(2), 117–134. https://doi.org/10.1016/j.sorms.2016.10.001
Mani, V., Delgado, C., Hazen, B. T., & Patel, P. (2017). Mitigating Supply Chain Risk via Sustainability Using Big Data Analytics: Evidence from the Manufacturing Supply Chain. Sustainability, 9(4), 608. https://doi.org/10.3390/su9040608
Meira, W., Jr. (2017). Fairness, Accountability, and Transparency While Mining Data from the Web and Social Networks. Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web, 17–17. https://doi.org/10.1145/3126858.3133314
Petersen, C. (2018). Through Patients’ Eyes: Regulation, Technology, Privacy, and the Future. Yearbook of Medical Informatics, 27(01), 010–015. https://doi.org/10.1055/s-0038-1641193
Prewave. (2017). Prewave Case Study: Seaport Closures.
Prewave. (2018a). Introducing Prewave. Retrieved from http://i2c.ec.tuwien.ac.at/wp-content/uploads/2016/07/Introducing_Prewave.pdf
Prewave. (2018b). Prewave | Artificial Intelligence Meets Risk Intelligence. Retrieved November 19, 2018, from Prewave | Artificial Intelligence Meets Risk Intelligence website: https://www.prewave.ai/insurance
Singh, A., Shukla, N., & Mishra, N. (2018). Social media data analytics to improve supply chain management in food industries. Transportation Research Part E: Logistics and Transportation Review, 114, 398–415. https://doi.org/10.1016/j.tre.2017.05.008
Terzi, D. S., Terzi, R., & Sagiroglu, S. (2015). A survey on security and privacy issues in big data. 2015 10th International Conference for Internet Technology and Secured Transactions (ICITST), 202–207. https://doi.org/10.1109/ICITST.2015.7412089
Torous, J., Larsen, M. E., Depp, C., Cosco, T. D., Barnett, I., Nock, M. K., & Firth, J. (2018). Smartphones, Sensors, and Machine Learning to Advance Real-Time Prediction and Interventions for Suicide Prevention: a Review of Current Progress and Next Steps. Current Psychiatry Reports, 20(7). https://doi.org/10.1007/s11920-018-0914-y
TU Wien. (2018). Vienna-based AI startup Prewave secures seed investment from IST Cube and Pioneers Ventures – Innovation Incubation Center. Retrieved November 19, 2018, from https://i2c.ec.tuwien.ac.at/vienna-based-ai-startup-prewave-secures-seed-investment-from-ist-cube-and-pioneers-ventures/
van der Aalst, W. M. P., Bichler, M., & Heinzl, A. (2017). Responsible Data Science. Business & Information Systems Engineering, 59(5), 311–313. https://doi.org/10.1007/s12599-017-0487-z
Wachter, S., Mittelstadt, B., & Floridi, L. (2017a). Transparent, explainable, and accountable AI for robotics. Science Robotics, 2(6), eaan6080. https://doi.org/10.1126/scirobotics.aan6080
Wachter, S., Mittelstadt, B., & Floridi, L. (2017b). Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation. International Data Privacy Law, 7(2), 76–99. https://doi.org/10.1093/idpl/ipx005
Waller, M. A., & Fawcett, S. E. (2013). Data Science, Predictive Analytics, and Big Data: A Revolution That Will Transform Supply Chain Design and Management. Journal of Business Logistics, 34(2), 77–84. https://doi.org/10.1111/jbl.12010
Wang, G., Gunasekaran, A., Ngai, E. W. T., & Papadopoulos, T. (2016). Big data analytics in logistics and supply chain management: Certain investigations for research and applications. International Journal of Production Economics, 176(C), 98–110.
Williams, A. M., Liu, Y., Regner, K. R., Jotterand, F., Liu, P., & Liang, M. (2018). Artificial intelligence, physiological genomics, and precision medicine. Physiological Genomics, 50(4), 237–243. https://doi.org/10.1152/physiolgenomics.00119.2017
Xia, Y., Liu, Y., & Chen, Z. (2013). Support Vector Regression for prediction of stock trend. 2013 6th International Conference on Information Management, Innovation Management and Industrial Engineering, 2, 123–126. https://doi.org/10.1109/ICIII.2013.6703098

Funding

This case study was part of the SHERPA project, which received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 786641.

Acknowledgements

The author acknowledges the contribution of Prewave and the SHERPA Project consortium to the development and design of the case study.

Copyright

Copyright remains with the authors. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.