<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1d1 20130915//EN" "JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" article-type="research-article" xml:lang="en"><front><journal-meta><journal-id journal-id-type="publisher-id">SDQ</journal-id><journal-title-group><journal-title>Security &amp; Defence Quarterly</journal-title><abbrev-journal-title>SDQ</abbrev-journal-title></journal-title-group><issn pub-type="epub">2544-994X</issn><issn pub-type="ppub">2300-8741</issn><publisher><publisher-name>Akademia Sztuki Wojennej</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="publisher-id">SDQ-49-00370</article-id><article-id pub-id-type="doi">10.35467/sdq/197309</article-id><article-categories><subj-group subj-group-type="heading"><subject>Security &amp; Defence Quarterly</subject></subj-group></article-categories><title-group><article-title>Crowdsourcing for security in the age of hybrid threats</article-title></title-group><contrib-group content-type="authors"><contrib contrib-type="author"><contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-9632-2914</contrib-id><name><surname>Dobák</surname><given-names>Imre</given-names></name><email>dobak.imre@uni-nke.hu</email></contrib><aff id="aff1">Institute of National Security, National University of Public Service, Ludovika tér, H-1083, Budapest, Hungary</aff></contrib-group><pub-date pub-type="epub"><day>31</day><month>01</month><year>2025</year></pub-date><volume>49</volume><issue>1</issue><fpage>1</fpage><lpage>14</lpage><history><date date-type="received"><day>20</day><month>05</month><year>2024</year></date><date date-type="rev-recd"><day>08</day><month>12</month><year>2024</year></date><date date-type="accepted"><day>12</day><month>12</month><year>2024</year></date></history><permissions><copyright-statement>© 2025 I. Dobák, published by War Studies University, Poland.</copyright-statement><copyright-year>2025</copyright-year><license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/4.0"><license-p>This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">http://creativecommons.org/licenses/by/4.0/</ext-link>).</license-p></license></permissions><abstract><p>The study explores emerging forms of crowdsourcing solutions in cyberspace for security and national security purposes. It aims to systematise the main types of crowdsourcing solutions that involve civilian participation and contribute to the overall security of society in an increasingly broader context. The study begins with international examples of crowdsourcing, which can be visible in various areas of security. Identifying their specific features as they appear on open platforms helps to frame and examine the issue. Nowadays, the multitude of international security conflicts and processes indicate that the use of information and communications technology tools available at the individual level can effectively support the (national) security use of knowledge and expertise available in a distributed manner as a concentrated resource. The examples examined in this study indicate that the crowdsourcing approach can play a significant role in security and national security strategies across various areas. However, the adoption and effectiveness of crowdsourcing solutions in the voluntary contribution of individuals to security will undoubtedly be influenced by the nature of security events and the open involvement of members of society.</p></abstract><kwd-group xml:lang="en"><kwd>intelligence</kwd><kwd>hybrid</kwd><kwd>cyberspace</kwd><kwd>information society</kwd><kwd>national security</kwd></kwd-group></article-meta></front><body><sec id="S1"><title>Introduction</title><p>Crowdsourcing, conceptually, can be defined as a distributed work process in which tasks are outsourced to participants (the crowd) who contribute based on their skills or opportunities, collaboratively solving tasks to produce a common result. This model can be found across various domains, from economics and engineering to technology and the digital, data-driven fabric of modern society. The relevance of crowdsourcing extends to the security sector, which, in the 21st century’s era of hybrid conflicts, is increasingly focused on maximising the potential of cyberspace; in particular, fields such as information warfare, which includes influencing members of society, deceiving adversaries through disinformation, and conducting cyberspace-related intelligence and information gathering. The phenomenon is now present on social platforms, widely used by society. On the one hand, they create an abundance of openly available information; on the other, they serve as platforms for deception and the dissemination of misinformation.</p><p>Crowdsourcing is used across a wide range of research areas and is evident in our everyday lives (<xref ref-type="bibr" rid="ref7">Castillo, 2013</xref>; <xref ref-type="bibr" rid="ref11">Gupta and Brooks, 2013</xref>, p. 27). Participants contribute their data, knowledge, or resources, such as financial support (e.g. crowdfunding), to the creation, improvement, or refinement of a collective “outcome.” It is present in information and communications technology (ICT), including the crowdsourcing of the Internet of things (<xref ref-type="bibr" rid="ref2">Anopa et al., 2023</xref>) as well as in business and marketing. It is also increasingly significant in the social sciences, where it has driven specific directions of innovation, such as geo-crowdsourcing and crowdfunding (<xref ref-type="bibr" rid="ref5">Brovelli et al., 2019</xref>, pp. 838–863; <xref ref-type="bibr" rid="ref8">Demiray et al., 2019</xref>, pp. 115–151).</p><p>However, “crowdsourcing” as a model is not a phenomenon unique to the 21st century. From a scientific perspective, crowdsourcing refers to the outsourcing of parts of work processes and human resources in a distributed manner (<xref ref-type="bibr" rid="ref34">Stottlemyre, 2015</xref>). It involves engaging crowds as distributed resources to perform various “subtasks,” which today, thanks to online space and communities, has evolved into a powerful distributed problem-solving capability (<xref ref-type="bibr" rid="ref4">Brabham, 2008</xref>, p. 76). The logical progression of this development is to obtain solutions or insights from actors in cyberspace who can provide relevant information, whether through professional expertise or their presence in specific contexts. The term “crowdsourcing” is often attributed to Jeff <xref ref-type="bibr" rid="ref14">Howe (2006)</xref>, who introduced it from a business perspective in a 2006 article. Today, however, distributed work based on cooperation is prevalent across various aspects of life, as numerous technological and infocommunication solutions are built upon shared resources. The rise of crowdsourcing has been propelled by the accessibility of infocommunication tools at the individual level, which has expanded the spatial boundaries of the information we can access and share. These tools essentially act as “sensors” of our daily lives.</p><p>The opinions and information shared on various social platforms can be particularly valuable in critical security situations, as they often provide more accurate and up-to-date information than traditional sources. However, beyond these ideal scenarios, questions may arise about the value and reliability of the information shared. What is undeniable is that during such periods (e.g. natural disasters or conflicts), there is an increased demand for accurate and credible information from both members of society and public organisations responsible for security. In this context, cyberspace-based crowdsourcing solutions can play a significant role in supporting the work of agencies responsible for ensuring security.</p><p>Based on examples from the international literature, the crowdsourcing approach to national security demonstrates the collective power of communities, whether in information gathering, analysis, or forecasting. In this context, gaining a deeper understanding of the potential applications of the method and continuously seeking ways to further develop it for security purposes is essential. The aim of this paper is to contribute to the scientific discourse on applying crowdsourcing to (national) security by highlighting some of its main directions, as identified in the literature and current international examples. The paper categorises the topic into four key areas: information gathering, cybersecurity, expert analysis, and security foresight. It examines the specifics of these application directions as well as the advantages and limitations of the method.</p></sec><sec id="S2"><title>Methods</title><p>A key element of this research is the concept of “collective intelligence”, which refers to the shared abilities and knowledge of group members. A fundamental aspect of collective intelligence is the ability and willingness to cooperate, where participants work together towards a common goal and share tasks among themselves. This concept is demonstrated in numerous international examples, which serve as a basis for this study. As an inductive research method, this paper examines the topic by drawing on case studies from the literature and incorporating the findings of other authors in the context of security-related applications. While crowdsourcing is now highly diverse, much of it is not directly related to security. Therefore, during the literature review, document analysis, and selection of examples, a focus was placed on identifying examples that are central and illustrative for security-related applications (e.g. <xref ref-type="bibr" rid="ref26">Papapesios et al., 2019</xref>). The reviewed sources and examples provided the basis for identifying key characteristics and elements of crowdsourcing in security contexts as well as for outlining its main application directions. The choice of research method reflects the observation that crowdsourcing, as a theory, is often intertwined with security-related activities. This study seeks to explore how the method can be applied within the paradigm of security, rather than treating it as a distinct, clearly delineated category.</p><p>Due to a lack of available data, this study did not aim to explore the inner workings and characteristics of crowdsourcing systems. Instead, it aimed to identify specificities along the lines of the categorisation of security-related applications. At the same time, given the broad interpretability of security, it is not possible to draw sharp distinctions between the categories identified in the study. However, the examples examined provide valuable illustrations of how crowdsourcing is applied in these security-related contexts. It is hoped that identifying the main application categories and outlining the advantages and disadvantages of the method will be beneficial for those considering crowdsourcing solutions. Additionally, the study aims to offer ideas that could facilitate the further development and application of crowdsourcing in security-related fields.</p><p>This research aligns with the growing research direction in applying crowdsourcing methods to security contexts (<xref ref-type="bibr" rid="ref12">Halder, 2014</xref>; <xref ref-type="bibr" rid="ref15">Hui, 2015</xref>; <xref ref-type="bibr" rid="ref20">Markowsky, 2013</xref>). The global information environment, characterised by the availability of vast amounts of data in various formats and platforms, opens new perspectives for security applications. For instance, during a security incident, the challenge of identifying and selecting relevant information of value becomes significant. This increases the importance of sources that can enable security organisations to act more accurately and quickly. From this perspective, crowdsourcing views crowd members as distributed resources whose significance lies in their skills or geographic location. In critical security incidents, identifying the fastest and most accurate source of information becomes crucial. On the one hand, individuals in the affected geographic area may possess relevant information, and available tools can facilitate the transmission of that data. However, the accuracy and authenticity of the data cannot be guaranteed, whether due to accidental errors or intentional deception. Therefore, the advantages and disadvantages of crowdsourcing as a method must be considered carefully.</p><p>In this regard, as a methodological element, it is important to examine the distinct components of crowdsourcing (the crowd, the crowdsourcer, the crowd-sourced task, and the crowdsourcing platform; <xref ref-type="bibr" rid="ref13">Hosseini et al., 2014</xref>, pp. 1–12). Additionally, consideration must be given to both advantages and disadvantages of crowdsourcing, including data reliability, as well as the motivation and willingness of participants to participate.</p></sec><sec id="S3"><title>Forms and characteristics of crowdsourcing for security purposes</title><p>The conflicts of recent years have increasingly valued crowdsourcing solutions for security purposes. This trend reveals an increasingly complex picture, encompassing areas such as intelligence, analysis, forecasting, cybersecurity, and cyberattacks. One key factor driving this development is the involvement of civil society actors in modern armed conflicts. In today’s conflicts, alongside traditional military forces, civilian participants are also engaged—often through invisible networks—on both defending and attacking sides. These actors contribute to the events by providing information and support within cyberspace.</p><p>In a national security context, a key objective is to obtain the most accurate and up-to-date understanding of a given event. Achieving this requires human resources capable of contributing their skills, expertise, and information to piece together a more complete picture from mosaic-like elements.</p><p>Aligned with these objectives, numerous international examples demonstrate the use of crowdsourcing for security and national security purposes. These efforts use the unique characteristics of the online space to their advantage, including:</p><p><list list-type="bullet"><list-item><p>the actors of the information environment engaged in the given task (either information collection or analysis), which can be used to enhance capabilities;</p></list-item><list-item><p>platforms can be developed to reduce the time required for obtaining or processing information;</p></list-item><list-item><p>diverse types of information can be systematically and targetedly collected and processed, particularly in connection with security incidents where local human resources possess knowledge. In such cases, the involvement of members of society can significantly improve efficiency.</p></list-item></list></p><p>On the other side of the coin, crowdsourcing also presents challenges and disadvantages, making it unsuitable as a “one-size-fits-all” solution:</p><p><list list-type="bullet"><list-item><p>Authenticity and accuracy of data provided by participants as well as the underlying motivations for their contributions.</p></list-item><list-item><p>Risks of distortion, manipulation, or contamination of the process through data.</p></list-item><list-item><p>Participant security during conflicts, as their openly shared data or mobile communication devices could be vulnerable to attack, especially due to their physical geo-location.</p></list-item></list></p><p>Based on the international examples reviewed, this study has grouped the main applications of crowdsourcing for security purposes into four clusters and seeks to highlight the relevance of the topic through their main characteristics (<xref ref-type="fig" rid="F1">Figure 1</xref>). The clusters are not rigidly defined, and their boundaries often overlap.</p><fig id="F1" orientation="portrait" position="float"><label>Figure 1.</label><caption><p>Possible main directions of crowdsourcing solutions for security purposes.</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://securityanddefence.pl/f/fulltexts/197309/SDQ-49-00370-g001_min.jpg" xlink:fullhref="https://securityanddefence.pl/f/fulltexts/197309/SDQ-49-00370-g001.jpg"/></fig><sec id="S3_1"><title>Crowdsourcing for information collection (crowdsource intelligence)</title><p>Among the general types of crowdsourcing, the categories of “audience-centric, geocentric, event-centric, and global” crowdsourcing (<xref ref-type="bibr" rid="ref9">Erickson, 2010</xref>) provide a useful framework for models applicable in today’s online environment. The widespread adoption of online tools has significantly accelerated information transmission, which has created a shift in how human resources, ICT tools, and openly available information interact. Infocommunication tools now enable individuals not only to receive information but also to actively participate in shaping processes by contributing their data. By connecting their smartphones to online platforms and social media, people can participate in larger, coordinated efforts to collect and geo-reference data, improving the accuracy of assessments in specific situations, including security-related events (<xref ref-type="bibr" rid="ref26">Papapesios et al., 2019</xref>). In the national security context, crowdsourced intelligence aligns closely with concepts like Human Intelligence (HUMINT) and OpenSource Intelligence (OSINT) due to the prominence of human involvement and open information (<xref ref-type="bibr" rid="ref34">Stottlemyre, 2015</xref>). However, the term OSINT may be misinterpreted in this context. Here, it refers to the potential openness of information contributed by the crowd, rather than the process itself. In many cases, the aggregated results of crowdsourcing are no longer openly accessible, such as in open source software development process (<xref ref-type="bibr" rid="ref4">Brabham, 2008</xref>, p. 81).</p><p>The main purpose of the following examples is to demonstrate the types of information-gathering applications of crowdsourcing.</p><p>One significant application is during natural or humanitarian disasters and armed conflicts. Data collected by information providers in the affected area—despite the method’s limitations (e.g. inaccuracies or incorrect information)—can directly support coordination efforts, crisis management, and even disaster resilience (<xref ref-type="bibr" rid="ref22">Moghadas et al., 2023</xref>). The value of such information lies in its immediacy and its origin from the field, offering authorities a more accurate understanding of the size and nature of a security incident. However, in crisis situations, it is evident that reliance solely on this information is not feasible due to its inherent limitations. Factors such as intentional or unintentional deception, panic, misinterpreted knowledge, or inaccuracies arising from irresponsible behaviour linked to a sense of digital anonymity can all compromise the reliability of crowdsourced data.</p><p>These application possibilities have long been recognised by researchers and professionals in the field (<xref ref-type="bibr" rid="ref12">Halder, 2014</xref>). Comprehensive solutions that use the capabilities of crowdsourcing communities include examples such as fire-signalling systems and the visualisation of data on maps. For example, Europol’s Stop Child Abuse—Trace an Object (SCATO) is a platform which engages the public to help identify objects in images associated with child abuse cases (<xref ref-type="bibr" rid="ref16">Ilbiz and Kaunert, 2023</xref>). Such applications support the work of law enforcement and government agencies by providing digital evidence and aiding in the subsequent investigation of incidents by authorities.</p><p>Engaging and involving members of society on online platforms during crimes and major security incidents (e.g. terrorist attacks or bombings) is not a new phenomenon. Publicly available information can aid in identifying perpetrators and detecting individual crimes or security-related cases. The relationship between social media and crowdsourcing has been investigated in several studies, with one of the best-known cases being the Boston Marathon bombing in 2013 (<xref ref-type="bibr" rid="ref24">Nhan et al., 2017</xref>, pp. 341–361; <xref ref-type="bibr" rid="ref36">Tapia and LaLone, 2014</xref>, pp. 61–77). While crowdsourcing intelligence has clear advantages, the Boston Marathon case also highlighted its errors and contradictions when applied to security purposes. Following the bombings, groups formed on social networking sites to assist in identifying potential perpetrators (<xref ref-type="bibr" rid="ref20">Markowsky, 2013</xref>, pp. 772–773). The “investigative community” created online provided valuable support to law enforcement and investigative authorities. However, these groups also demonstrated the risks of crowdsourcing, such as the reinforcement of false connections and the wrongful identification of innocent individuals as suspects (<xref ref-type="bibr" rid="ref21">McCullagh, 2013</xref>). Despite the authorities’ calls for voluntary societal cooperation to expand the evidence base, self-organised online groups were able to call innocent people as suspects. This also illustrated the limitations and potential failures of crowdsourcing when used for security purposes. A more recent example of crowdsourcing intelligence (CSI) occurred during the violent events at the United States Capitol in January 2021. Crowds helped law enforcement officers identify insurgents by analysing photos and videos posted on social media (<xref ref-type="bibr" rid="ref39">Zegart, 2021</xref>).</p><p>The Russian-Ukrainian war provides examples of the wide-ranging security-related applications of crowdsourcing. From the outset of the war, Ukraine recognised the value of social media platforms and smartphone-generated information to track and report the activities of Russian forces. However, this example also highlights the disadvantages of crowdsourcing in security contexts, such as concerns about the safety of those sharing information, the potential inaccuracy of data, and deliberate disinformation efforts. Ukrainian security agencies quickly leveraged the crowdsourcing method. For instance, the Ukrainian Security Service created the STOP Russian War chatbot, which enabled citizens to report enemy movements and activities. The Diia (e-governance digital portal) initially played a key role in enhancing the credibility of information and the effectiveness of crowdsourcing efforts. However, the system later became a target for Russian spambots, which required the implementation of additional protective measures (<xref ref-type="bibr" rid="ref3">Bergengruen, 2022</xref>; <xref ref-type="bibr" rid="ref6">Burke, 2022</xref>, p. 96). These challenges (data poisoning, cyberattacks, and system overloads during critical periods) point at problematic aspects of crowdsourcing for security purposes.</p><p>Other notable Ukrainian initiatives include the Narodnii Mesnik platform, launched by the Ukrainian National Police to collect signals from the public, and the eVorog (e-enemy) Telegram chatbot, launched by the Ministry of Digital Transformation, to detect and monitor Russian military activities. Additionally, the online project Russia Will Pay aims to document and assess war damages (<xref ref-type="bibr" rid="ref28">Russia Will Pay, 2024</xref>). Organised by the Kyiv School of Economics (KSE Institute), the Office of the President of Ukraine, and several ministries, the project’s goal is to control, analyse, and estimate the material damage to Ukraine’s infrastructure and provide detailed proof for accountability. According to the KSE Institute’s website, the program guarantees the confidentiality of individuals who submit information.</p><p>A common feature of the above crowdsourcing solutions is their direct connection to security issues. These solutions rely on the participation of the civilian population, who provide relevant information (human resources), and on the technological environment, which is frequently linked to external infocommunications development efforts. Additionally, they often involve actors from the scientific community.</p></sec><sec id="S3_2"><title>Crowdsourcing for cybersecurity</title><p>In this form, crowdsourcing models rely on the expertise, specific knowledge, and abilities of participants. Their applications in security-related contexts are highly diverse, ranging from the processing and evaluation of specialised data to addressing the rapidly evolving challenges in the field of cybersecurity.</p><p>In the field of cybersecurity, the crowdsourcing model is a well-established approach, particularly used by software development companies for activities, such as Crowdsourced Software Testing (CST) (<xref ref-type="bibr" rid="ref1">Alyahya, 2020</xref>). One prominent application is in business-driven bug bounty programs, where ethical hacker communities (whether public or private) assist in testing software and enhancing information security by identifying vulnerabilities. The primary objective of such programs is to engage individuals with expertise in the field, such as ethical hackers, researchers, and developers, who can identify cybersecurity vulnerabilities either periodically during specific campaigns or on a continuous basis.</p><p>In this context, crowdsourcing can be employed by companies and organisations to gather and share information through platforms such as threat exchange systems, thereby contributing to the development of necessary countermeasures (<xref ref-type="bibr" rid="ref27">Pawar, 2023</xref>). However, despite its advantages, these data-sharing solutions are often not viable due to the sensitivity of the data involved.</p><p>Examples of crowdsourced cybersecurity applications include citizen volunteering initiatives, which use the Internet to share information and fight cybercrime and cyber threats. One such example is the CyberPeace Corps (<xref ref-type="bibr" rid="ref18">Kumar, 2022</xref>). However, crowdsourced methods can also be observed from the perspective of attackers.</p><p>In contemporary armed conflicts, such as the Russian-Ukrainian war and the Israel-Hamas war, groups have emerged on both sides to attack their adversaries’ systems in cyberspace. These groups often employ crowdsourced Distributed Denial of Service (DDoS) attack methods. <xref ref-type="bibr" rid="ref25">Osta’s study (2024)</xref> provides a detailed account of these activities, describing the actions of entities like the IT Army of Ukraine, NoName057(16), and the Cyber Army of Palestine. The study examines the crowdsourced DDoS attack methods they use and the traceability of their effects for anonymised participants. The study also highlights that organised solutions for disrupting systems and services in cyberspace are now readily available as open-source tools, the real catalyst of which is the “crowd” that uses them.</p><p>The specificity of this category lies in the distributed expertise and the associated involvement of human resources, often complemented by additional motivating factors, such as financial incentives, beyond mere volunteering. However, the level of expertise required for participation can be reduced through supportive interfaces (e.g. for cyber-attacks or analysis), which can facilitate the involvement of a broader range of participants.</p></sec><sec id="S3_3"><title>Crowdsourcing for expert analysis</title><p>The use of crowdsourcing for expert purposes is also evident in the field of security. One notable example is the case of Malaysia Airlines flight MH370, which disappeared in 2014 during its journey from Kuala Lumpur to Beijing. Millions of participants joined the platform Tomnod to review over 1 million square kilometres of high-resolution satellite images in the search for clues. Another example comes from Germany’s Every Name Counts project, based on the Arolsen Archives. Citizens contributed to the expert processing of digitised records related to the victims of Nazi persecution by indexing them into a searchable database. As a result of social media engagement, 7,000 volunteers joined the project in 2020 (<xref ref-type="bibr" rid="ref10">Ghert-Zand, 2020</xref>). A further example is the Texas Virtual Border Watch program, a government-led web-based monitoring solution developed by the US law enforcement agencies. This initiative allows citizens to participate in monitoring the US–Mexico border, aiding efforts to combat illegal activities, smuggling, and unauthorised immigration (<xref ref-type="bibr" rid="ref37">Tewksbury, 2012</xref>, p. 250).</p><p>As in the preceding category, the crowd’s skills are used, but the distinction lies in participants being tasked with analysing already available information. The rationale for using this model stems from two key factors: the substantial volume of data that needs to be processed and the necessity to accomplish this within a constrained timeframe. Meanwhile, the range of applications for this method continues to expand. For instance, recent research (<xref ref-type="bibr" rid="ref17">Jia et al., 2024</xref>) examines the role of collaborative groups in identifying online misinformation through the “wisdom of crowds,” where the aggregate opinion of the group proves more accurate than that of individuals (<xref ref-type="bibr" rid="ref32">Simoiu et al., 2019</xref>).</p></sec><sec id="S3_4"><title>Crowdsourcing for security forecast purposes (crowdsourced forecasting or CSF)</title><p>The “ability to predict” is an important focus in defence and security planning that can be enhanced through the use of large datasets or by using the knowledge and skills of individuals and organisations. Crowdsourced forecasting is gaining importance as a method for anticipating future events by aggregating the opinions and expertise of the crowd, such as in technology assessments. Scientific research on CSF for national security purposes seeks to support policymaking by predicting future challenges and risks. While open solutions are commonly applied to security issues, the literature suggests that national security forecasting research increasingly leans towards closed, internal systems (<xref ref-type="bibr" rid="ref29">Samotin et al., 2023</xref>). However, the true power of crowdsourcing lies in its capacity to engage the widest possible range of participants who can make meaningful contributions to the success of a programme. In many cases, this potential justifies extending the reach of such programmes and opening them up to other contributors.</p><p>A study examining the relationship between the US Intelligence Community (IC) and crowdsourcing (<xref ref-type="bibr" rid="ref29">Samotin et al., 2023</xref>) highlights the development of two significant crowdsourcing platforms over the past decade: the Intelligence Community Prediction Market (ICPM) and the Aggregative Contingents Estimation Program (ACE). These platforms have demonstrated the potential to provide effective forecasting solutions. However, as the study explains, despite their promise, the two programs have not had a substantial impact on the US intelligence reporting. To address this, the research explores several possible reasons for their limited influence. According to the study, the Intelligence Advanced Research Projects Activity (IARPA) launched the US Intelligence Community’s classified network prediction market, which has been cited in another study (<xref ref-type="bibr" rid="ref33">Stastny and Lehner, 2018</xref>) as being “more accurate than predictions made in traditional intelligence reports” (<xref ref-type="bibr" rid="ref29">Samotin et al., 2023</xref>, p. 558).</p><p>The George Mason University SciCast project, funded by IARPA and launched a decade ago, serves as another example of research into the potential for predicting security threats. According to <xref ref-type="bibr" rid="ref30">SciCast, 2024</xref>, the funding for this research has ended, and the website is currently inactive. The project aimed to explore the use of group-level collective wisdom to predict future events related to national security. As explained in an article by <xref ref-type="bibr" rid="ref38">Tucker (2014)</xref>, participants in the project used their expertise and knowledge to bet on the likelihood of future events.</p><p>Other programs fall into this category, such as the US Integrated Forecasting and Estimates of Risk (INFER) program, which enhances government forecasting capabilities by using crowdsourced sources to predict future scientific, technological, and national security trends and processes. Since 2023, INFER has collaborated with the Cosmic Bazaar program, led by the UK’s Professional Head of Intelligence Assessment (<xref ref-type="bibr" rid="ref31">Siegel, 2023</xref>).</p><p>The specificity of this category lies in the fact that it does not rely on existing data or information as a starting point. Instead, it is based on the crowd’s intuitions, opinions, and predictions about future developments. In this context, inaccuracies in the input provided by participants cannot be classified as data pollution.</p></sec></sec><sec id="S4"><title>Results</title><p>The examined examples suggest that the crowdsourcing approach has potential applications in various aspects of security and national security thinking. However, both limitations and advantages of the method must be considered (see <xref ref-type="fig" rid="F2">Figure 2</xref>). While national security thinking is often “closed” in many areas, critical events, such as terrorist attacks or open military conflicts, create a need for information flows that extend to society. These information needs are particularly apparent in the areas where communication and collaboration with broader social groups can support security interests. Examples include assistance during natural and man-made disasters, law enforcement efforts, or the dissemination of accurate information during armed conflicts (<xref ref-type="bibr" rid="ref22">Moghadas et al., 2023</xref>; <xref ref-type="bibr" rid="ref26">Papapesios et al., 2019</xref>).</p><fig id="F2" orientation="portrait" position="float"><label>Figure 2.</label><caption><p>Limitations and benefits of crowdsourcing solutions for security.</p></caption><graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://securityanddefence.pl/f/fulltexts/197309/SDQ-49-00370-g002_min.jpg" xlink:fullhref="https://securityanddefence.pl/f/fulltexts/197309/SDQ-49-00370-g002.jpg"/></fig><p>Regarding the advantages of the crowdsourcing method, several general benefits can be identified. These include a reduction in reaction time, an increase in the volume of information (and the anticipated accuracy), and economic, social, and human advantages. Other authors highlight the power of the crowd, emphasising the utilisation of human resources and expertise, the potential for discreet work, cost savings, and even fostering innovation (<xref ref-type="bibr" rid="ref15">Hui, 2015</xref>, p. 5). However, crowdsourcing is not a universal solution for addressing security issues. Applications for security purposes raise specific questions, such as whether participant anonymity is necessary or how to ensure data authenticity and accuracy (<xref ref-type="bibr" rid="ref16">Ilbiz and Kaunert, 2023</xref>). By definition, it depends on the purpose of the crowdsourcing application, whether it is necessary to ensure the anonymity of the individual contributing by solving partial tasks and providing partial information as part of the “anonymous” crowd. In one case, for example, we can think of using a route planner, where we contribute with our efforts to determine the amount of traffic, while in another case we want to contribute to increasing security by handing over digital evidence (“anonymous reporting”).</p><p>Another critical aspect is that many security-related applications are connected to public organisations and tasks, which often raises concerns about personal data protection and privacy. These issues can significantly influence public motivation and willingness to participate (<xref ref-type="bibr" rid="ref19">Kumar and Faisal, 2024</xref>). For example, the COVID-tracking applications faced resistance due to fears about potential privacy violations, which acted as a restraining force (<xref ref-type="bibr" rid="ref23">Németh and Magyar, 2021</xref>). Similarly, the open use of social media platforms for security purposes, as seen during the Russian-Ukrainian war, has highlighted similar concerns regarding data privacy and user anonymity.</p><p>Social platforms offer a viable medium for expanding crowdsourcing initiatives. However, in security-related topics, it is crucial to ensure both authenticity of the information and safety (including anonymity) of the participants. Due to their openness, such platforms are particularly vulnerable to influence, manipulation, and the dissemination of disinformation, which can undermine the authenticity of “community” knowledge. Community-sourced data also raises concerns about accuracy and credibility. While inaccuracies may result from innocent errors, they can also stem from deliberate actions, such as intentional deception.</p><p>The phenomenon of data poisoning, where malicious participants deliberately contaminate data collection with misleading information, poses a significant challenge, particularly in security applications. This contamination can negatively impact the accuracy of aggregated results, potentially leading to erroneous conclusions and decisions (<xref ref-type="bibr" rid="ref35">Tahmasebian et al., 2020</xref>). In security-related contexts, where the data collected may directly influence societal safety, the intentional manipulation of information is especially concerning. Such malicious and intentional actions can lead to erroneous conclusions and decisions and can undermine trust in these applications. In today’s world, where data analysis and evaluation are increasingly reliant on artificial intelligence solutions, the importance of “authentic” data is crucial, as inaccurate data can distort the entire analysis process.</p><p>As a potential solution, double-checking smaller and refined datasets that emerge as valuable during the crowdsourcing process (e.g. through expert validation) can help mitigate inaccuracies. Additionally, leveraging the collective effect of mass confirmation (where multiple participants validate the same information) can neutralise the impact of inaccurate data. While this solution is more feasible in crowdsourcing intelligence applications, where data can be verified against existing facts or expert analysis, it presents a challenge in crowdsourcing forecasting. In forecasting scenarios, the accuracy of predictions can only be verified retrospectively after the predicted event has occurred.</p></sec><sec id="S5"><title>Conclusions</title><p>Crowdsourcing has become an integral yet often invisible part of our daily lives. Its relevance to security has been amplified by several factors. First, the evolution of information technology (IT) tools and environments has expanded the potential for crowdsourcing applications. Second, security authorities have recognised the value of mobile communication tools, which are essential for individuals to share information globally. These tools can generate relevant security-related information, which makes them a valuable asset for public authorities. Mobile devices can serve as a vital data feed. However, when designing and implementing such targeted applications, there are also a number of other aspects to consider.</p><p>The study identified four areas of application (namely information collection, expert use, cybersecurity, and security forecasting) where the crowdsourcing model is already demonstrating practical relevance to national security. These areas provide valuable examples and insights that can help shape the future use of crowdsourcing and contribute to the broader discourse on the subject.</p><p>The common elements underlying these types of applications include:</p><p><list list-type="bullet"><list-item><p>the necessity for involvement and voluntary participation,</p></list-item><list-item><p>the motivated engagement of participants,</p></list-item><list-item><p>the utilisation of cyberspace platforms, and</p></list-item><list-item><p>the requirement for a central organising role.</p></list-item></list></p><p>However, alongside the advantages of the method outlined in the study, there are several limitations that must be acknowledged. These challenges may stem from the participants themselves (e.g. the need for anonymisation, the reliability and credibility of data contributors) or from the datasets generated through their participation (e.g. data accuracy, credibility, and the security of the systems). It is therefore essential for developers of crowdsourcing-based applications to be mindful of these constraints and to address them during the design process. Ensuring robust safeguards and mechanisms to mitigate these challenges is critical to the effective applicability of the method. Failure to do so could result in cumulative community knowledge producing erroneous results.</p><p>What can be expected of the future? The nature of future security incidents and the associated open involvement of societal actors will undoubtedly shape the evolution of crowdsourcing security applications. The capacity to disseminate information through communication channels—whether in the context of ongoing conflicts, unforeseen natural or man-made disasters, or the complex domain of cybersecurity—will increasingly highlight the value of human involvement organised through crowdsourcing processes. Nevertheless, the role of civil society members as observers or external experts opens up numerous research avenues worthy of further exploration.</p></sec></body><back><sec id="S6" sec-type="financial-disclosure"><title>Funding</title><p>This research received no external funding.</p></sec><sec id="S7"><title>Data Availability Statement</title><p>Not applicable.</p></sec><sec id="S8" sec-type="COI-statement"><title>Disclosure Statement</title><p>No potential conflict of interest was reported by the author. The author read and agreed to the published version of the manuscript.</p></sec><ref-list><ref id="ref1"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Alyahya</surname>, <given-names>S</given-names></string-name></person-group>. (<year>2020</year>) <article-title>‘Crowdsourced software testing: A systematic literature review’</article-title>, <source>Information and Software Technology</source>, <volume>127</volume>, p. <fpage>106363</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.infsof.2020.106363</pub-id>.</mixed-citation></ref><ref id="ref2"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Anopa</surname>, <given-names>S</given-names></string-name>., <string-name><surname>Salim</surname>, <given-names>A</given-names></string-name>. and <string-name><surname>Pankaj</surname>, <given-names>D.S</given-names></string-name></person-group>. (<year>2023</year>) <article-title>‘Crowdsourcing of Internet of things: Applications, trends in technology and the future’, in Conference Proceedings: <italic>2023 International conference on power, instrumentation, control and computing (PICC)</italic></article-title>, <source>IEEE</source>, <publisher-name>Thrissur, India</publisher-name>, pp. <fpage>1</fpage>–<lpage>6</lpage>. doi: <pub-id pub-id-type="doi">10.1109/PICC57976.2023.10142617</pub-id>.</mixed-citation></ref><ref id="ref3"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Bergengruen</surname>, <given-names>V</given-names></string-name></person-group>. (<year>2022</year>) <article-title><italic>How Ukraine is crowdsourcing digital evidence of war crimes</italic></article-title>, <day>18</day><month>April</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://time.com/6166781/ukraine-crowdsourcing-war-crimes/">https://time.com/6166781/ukraine-crowdsourcing-war-crimes/</ext-link> (Accessed: 10 May 2024)</comment>.</mixed-citation></ref><ref id="ref4"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Brabham</surname>, <given-names>D.C</given-names></string-name></person-group>. (<year>2008</year>) <article-title>‘Crowdsourcing as a model for problem solving: An introduction and cases’</article-title>, <source>Convergence</source>, <volume>14</volume>(<issue>1</issue>), pp. <fpage>75</fpage>–<lpage>90</lpage>. doi: <pub-id pub-id-type="doi">10.1177/1354856507084420</pub-id>.</mixed-citation></ref><ref id="ref5"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Brovelli</surname>, <given-names>M.A</given-names></string-name>., <string-name><surname>Delipetrev</surname>, <given-names>B</given-names></string-name>. and <string-name><surname>Zamboni</surname>, <given-names>G</given-names></string-name></person-group>. (<year>2019</year>) <article-title>‘Free and open source tools for volunteer geographic information and geo-crowdsourcing’</article-title>, in <person-group person-group-type="editor"><collab>Information Resources Management Association</collab> (ed.)</person-group><source><italic>Crowdsourcing: Concepts, methodologies, tools, and applications</italic></source>. <publisher-loc>Hershey, PA</publisher-loc>: <publisher-name>IGI Global</publisher-name>, pp. <fpage>838</fpage>–<lpage>863</lpage>. doi: <pub-id pub-id-type="doi">10.4018/978-1-5225-8362-2.ch041</pub-id>.</mixed-citation></ref><ref id="ref6"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Burke</surname>, <given-names>P</given-names></string-name></person-group>. (<year>2022</year>) <article-title>‘The issues in the collection, verification and actionability of citizen-derived and crowdsourced intelligence during the Russian invasion of Ukraine’</article-title>, <source>Strategic Panorama, Special Issue</source>, pp. <fpage>94</fpage>–<lpage>103</lpage>. doi: <pub-id pub-id-type="doi">10.53679/2616-9460.specialissue.2022.09</pub-id>.</mixed-citation></ref><ref id="ref7"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Castillo</surname>, <given-names>M</given-names></string-name></person-group>. (<year>2013</year>) <article-title>‘The wisdom of crowds’</article-title>, <source>American Journal of Neuroradiology</source>, <volume>34</volume>(<issue>10</issue>), pp. <fpage>1863</fpage>–<lpage>1865</lpage>. doi: <pub-id pub-id-type="doi">10.3174/ajnr.A3417</pub-id>.</mixed-citation></ref><ref id="ref8"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Demiray</surname>, <given-names>M</given-names></string-name>., <string-name><surname>Burnaz</surname>, <given-names>S</given-names></string-name>., and <string-name><surname>Aslanbay</surname>, <given-names>Y</given-names></string-name></person-group>. (<year>2019</year>) <article-title>‘The crowdfunding market, models, platforms, and projects’</article-title>, in <person-group person-group-type="editor"><collab>Information Resources Management Association</collab> (ed.)</person-group><source><italic>Crowdsourcing: Concepts, methodologies, tools, and applications</italic></source>. <publisher-loc>Hershey, PA</publisher-loc>: <publisher-name>IGI Global</publisher-name>, pp. <fpage>115</fpage>–<lpage>151</lpage>. doi: <pub-id pub-id-type="doi">10.4018/978-1-5225-8362-2.ch007</pub-id>.</mixed-citation></ref><ref id="ref9"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Erickson</surname>, <given-names>T</given-names></string-name></person-group>. (<year>2010</year>) <chapter-title>‘Geocentric crowdsourcing and smarter cities: Enabling urban intelligence in cities and regions’, position paper presented at the <italic>1st Ubiquitous Crowdsourcing Workshop, UbiComp</italic></chapter-title>.</mixed-citation></ref><ref id="ref10"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Ghert-Zand</surname>, <given-names>R</given-names></string-name></person-group>. (<year>2020</year>) <article-title><italic>You can help Nazi victims’ families learn their fates in online archive project</italic></article-title>, <day>22</day><month>July</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.timesofisrael.com/you-can-help-nazi-victims-families-learn-their-fates-in-online-archive-project/">https://www.timesofisrael.com/you-can-help-nazi-victims-families-learn-their-fates-in-online-archive-project/</ext-link> (Accessed: 5 June 2024)</comment>.</mixed-citation></ref><ref id="ref11"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Gupta</surname>, <given-names>R</given-names></string-name>. and <string-name><surname>Brooks</surname>, <given-names>H</given-names></string-name></person-group>. (<year>2013</year>) <article-title><italic>Using social media for global security</italic></article-title>. <publisher-loc>Indianapolis</publisher-loc>: <publisher-name>John Wile</publisher-name>. <comment>Available at: Inc <ext-link ext-link-type="uri" xlink:href="https://books.google.co.in/books?id=Fm0uWf9EL7cC&amp;printsec=frontcover#v=onepage&amp;q&amp;f=false">https://books.google.co.in/books?id=Fm0uWf9EL7cC&amp;printsec=frontcover#v=onepage&amp;q&amp;f=false</ext-link> (Accessed: 10 April 2024)</comment>.</mixed-citation></ref><ref id="ref12"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Halder</surname>, <given-names>B</given-names></string-name></person-group>. (<year>2014</year>) <article-title>‘Crowdsourcing collection of data for crisis governance in the post-2015 world: Potential offers and crucial challenges’, in <italic>Proceedings of the 8th international conference on theory and practice of electronic governance</italic> (ICEGOV ‘14)</article-title>. <source>New York</source>, <publisher-loc>NY</publisher-loc>: <publisher-name>Association for Computing Machinery (ACM)</publisher-name>, pp. <fpage>1</fpage>–<lpage>10</lpage>. doi: <pub-id pub-id-type="doi">10.1145/2691195.2691208</pub-id>.</mixed-citation></ref><ref id="ref13"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Hosseini</surname>, <given-names>M</given-names></string-name>., <string-name><surname>Phalp</surname>, <given-names>K</given-names></string-name>., <string-name><surname>Taylor</surname>, <given-names>J</given-names></string-name>. and <string-name><surname>Ali</surname><given-names>R</given-names></string-name></person-group>. (<year>2014</year>) <article-title>‘The four pillars of crowdsourcing: A reference model’, in Conference Proceedings: <italic>2014 IEEE eighth international conference on research challenges in information science (RCIS)</italic></article-title>, <source>IEEE, Marrakech, Morocco</source>, <volume>2014</volume>, pp. <fpage>1</fpage>–<lpage>12</lpage>, doi: <pub-id pub-id-type="doi">10.1109/RCIS.2014.6861072</pub-id>.</mixed-citation></ref><ref id="ref14"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Howe</surname>, <given-names>J</given-names></string-name></person-group>. (<year>2006</year>) <article-title>‘The rise of crowdsourcing’, <italic>Wired</italic></article-title>, <day>1</day><month>June</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.wired.com/2006/06/crowds/">https://www.wired.com/2006/06/crowds/</ext-link> (Accessed: 10 April 2024)</comment>.</mixed-citation></ref><ref id="ref15"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Hui</surname>, <given-names>J.Y</given-names></string-name></person-group>. (<year>2015</year>) <article-title><italic>Crowdsourcing for national security</italic></article-title>. <source>Policy report</source>. <publisher-loc>Singapore</publisher-loc>: <publisher-name>S. Rajaratnam School of International Studies, Nanyang Technological University</publisher-name>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="http://www.jstor.org/stable/resrep05853">http://www.jstor.org/stable/resrep05853</ext-link></comment></mixed-citation></ref><ref id="ref16"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Ilbiz</surname>, <given-names>E</given-names></string-name>. and <string-name><surname>Kaunert</surname>, <given-names>C</given-names></string-name></person-group>. (<year>2023</year>) <article-title>‘Crowdsourcing to tackle online child sexual exploitation: Europol’s “stop child abuse—trace an object” platform’</article-title>, <source>Policing: A Journal of Policy and Practice</source>, <volume>17</volume><fpage>paad009</fpage>, doi: <pub-id pub-id-type="doi">10.1093/police/paad009</pub-id>.</mixed-citation></ref><ref id="ref17"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Jia</surname>, <given-names>C</given-names></string-name>., <string-name><surname>Lee</surname>, <given-names>A.Y</given-names></string-name>., <string-name><surname>Moore</surname>, <given-names>R.C</given-names></string-name>., <string-name><surname>Decatur</surname>, <given-names>C.H.-S</given-names></string-name>., <string-name><surname>Liu</surname>, <given-names>S.X</given-names></string-name>. and <string-name><surname>Hancock</surname>, <given-names>J.T</given-names></string-name></person-group>. (<year>2024</year>) <article-title>‘Collaboration, crowdsourcing, and misinformation’</article-title>, <source>PNAS Nexus</source>, <volume>3</volume>(<issue>10</issue>), p. <fpage>434</fpage>. doi: <pub-id pub-id-type="doi">10.1093/pnasnexus/pgae434</pub-id>.</mixed-citation></ref><ref id="ref18"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Kumar</surname>, <given-names>V</given-names></string-name></person-group>. (<year>2022</year>) <chapter-title>‘Crowdsourcing cyber peace and cybersecurity’</chapter-title>, in <person-group person-group-type="editor">4<string-name><surname>Shackelford</surname>, <given-names>S.J.</given-names></string-name>, <string-name><surname>Douzet</surname>, <given-names>F</given-names></string-name>. and <string-name><surname>Ankersen</surname>, <given-names>C</given-names></string-name>. (eds.)</person-group><source>Cyber peace: charting a path toward a sustainable, stable, and secure cyberspace</source>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>, pp. <fpage>230</fpage>–<lpage>235</lpage>.</mixed-citation></ref><ref id="ref19"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Kumar</surname>, <given-names>S</given-names></string-name>. and <string-name><surname>Faisal</surname>, <given-names>M</given-names></string-name></person-group>. (<year>2024</year>) <article-title>‘A comprehensive examination of digital privacy in crowdsourcing applications’</article-title>, <source><italic>2024 International conference on automation and computation (AUTOCOM)</italic></source>, <publisher-loc>Dehradun, India</publisher-loc>, pp. <fpage>352</fpage>–<lpage>357</lpage>. doi: <pub-id pub-id-type="doi">10.1109/AUTOCOM60220.2024.10486102</pub-id>.</mixed-citation></ref><ref id="ref20"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Markowsky</surname>, <given-names>G</given-names></string-name></person-group>. (<year>2013</year>) <article-title>‘Crowdsourcing, big data and homeland security’, in <italic>IEEE international conference on technologies for homeland security</italic> (HST)</article-title><volume>2013</volume>, pp. <fpage>772</fpage>–<lpage>778</lpage>. doi: <pub-id pub-id-type="doi">10.1109/THS.2013.6699101</pub-id>.</mixed-citation></ref><ref id="ref21"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>McCullagh</surname>, <given-names>D</given-names></string-name></person-group>. (<year>2013</year>) <article-title><italic>FBI seeks crowdsourcing help in Boston bombing case</italic>: <italic>ID these two men!</italic></article-title>, <day>18</day><month>April</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.cnet.com/tech/tech-industry/fbi-seeks-crowdsourcing-help-in-boston-bombing-case-id-these-two-men/">https://www.cnet.com/tech/tech-industry/fbi-seeks-crowdsourcing-help-in-boston-bombing-case-id-these-two-men/</ext-link> (Accessed: 05 May 2024)</comment>.</mixed-citation></ref><ref id="ref22"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Moghadas</surname>, <given-names>M</given-names></string-name>., <string-name><surname>Fekete</surname>, <given-names>A</given-names></string-name>., <string-name><surname>Rajabifard</surname>, <given-names>A</given-names></string-name>. and <string-name><surname>Kötter</surname>, <given-names>T</given-names></string-name></person-group> (<year>2023</year>) <article-title>‘The wisdom of crowds for improved disaster resilience: A near-real-time analysis of crowdsourced social media data on the 2021 flood in Germany’</article-title>. <source>GeoJournal</source>, <volume>88</volume>, pp. <fpage>4215</fpage>–<lpage>4241</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10708-023-10858-x</pub-id>.</mixed-citation></ref><ref id="ref23"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Németh</surname>, <given-names>A</given-names></string-name>. and <string-name><surname>Magyar</surname>, <given-names>S</given-names></string-name></person-group>. (<year>2021</year>) <article-title>‘An investigation of data used to support contact tracing to curb the spread of COVID-19 pandemic from the aspect of possible national security application (part 2)’</article-title>, <source>National Security Review: <italic>Periodical of the Military National Security Service</italic></source>, <volume>1</volume>(<issue>14</issue>), pp. <fpage>218</fpage>–<lpage>231</lpage>.</mixed-citation></ref><ref id="ref24"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Nhan</surname>, <given-names>J</given-names></string-name>., <string-name><surname>Huey</surname>, <given-names>L</given-names></string-name>. and <string-name><surname>Broll</surname>, <given-names>R</given-names></string-name></person-group>. (<year>2017</year>) <article-title>‘Digilantism: An analysis of crowdsourcing and the Boston Marathon bombings’</article-title>, <source>The British Journal of Criminology</source>, <volume>57</volume>(<issue>2</issue>), pp. <fpage>341</fpage>–<lpage>361</lpage>, doi: <pub-id pub-id-type="doi">10.1093/bjc/azv118</pub-id></mixed-citation></ref><ref id="ref25"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Osta</surname>, <given-names>Z</given-names></string-name></person-group>. (<year>2024</year>) <article-title><italic>Crowdsourced DDoS attacks amid geopolitical events</italic></article-title>. <day>16</day><month>January</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://flare.io/learn/resources/crowdsourced-ddos-attacks-amid-geopolitical-events/">https://flare.io/learn/resources/crowdsourced-ddos-attacks-amid-geopolitical-events/</ext-link> (Accessed: 10 May 2024).</comment></mixed-citation></ref><ref id="ref26"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Papapesios</surname>, <given-names>N</given-names></string-name>., <string-name><surname>Ellul</surname>, <given-names>C</given-names></string-name>., <string-name><surname>Shakir</surname>, <given-names>A</given-names></string-name>. and <string-name><surname>Hart</surname>, <given-names>G</given-names></string-name></person-group>. (<year>2019</year>) <article-title>‘Exploring the use of crowdsourced geographic information in defence: Challenges and opportunities’</article-title>, <source>Journal of Geographical Systems</source>, <volume>21</volume>, pp. <fpage>133</fpage>–<lpage>160</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10109-018-0282-5</pub-id>.</mixed-citation></ref><ref id="ref27"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Pawar</surname>, <given-names>S</given-names></string-name></person-group>. (<year>2023</year>) <article-title><italic>The power of collective intelligence: Leveraging threat intelligence to protect against cyber threats</italic></article-title>, <day>30</day><month>May</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.eccouncil.org/cybersecurity-exchange/threat-intelligence/leveraging-threat-intelligence-to-protect-against-cyber-threats/">https://www.eccouncil.org/cybersecurity-exchange/threat-intelligence/leveraging-threat-intelligence-to-protect-against-cyber-threats/</ext-link> (Accessed: 28 November 2024)</comment>.</mixed-citation></ref><ref id="ref28"><mixed-citation publication-type="web"><person-group person-group-type="author"><collab>Russia Will Pay</collab></person-group> (<year>2024</year>) <article-title><italic>Kyiv School of Economics</italic></article-title>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://kse.ua/russia-will-pay/">https://kse.ua/russia-will-pay/</ext-link> (Accessed: 15 May 2024).</comment></mixed-citation></ref><ref id="ref29"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Samotin</surname>, <given-names>L.R</given-names></string-name>., <string-name><surname>Friedman</surname>, <given-names>J.A</given-names></string-name>. and <string-name><surname>Horowitz</surname>, <given-names>M.C</given-names></string-name></person-group>. (<year>2023</year>) <article-title>‘Obstacles to harnessing analytic innovations in foreign policy analysis: A case study of crowdsourcing in the US intelligence community’</article-title>, <source>Intelligence and National Security</source>, <volume>38</volume>(<issue>4</issue>), pp. <fpage>558</fpage>–<lpage>575</lpage>. doi: <pub-id pub-id-type="doi">10.1080/02684527.2022.2142352</pub-id>.</mixed-citation></ref><ref id="ref30"><mixed-citation publication-type="web"><person-group person-group-type="author"><collab>SciCast</collab></person-group> (<year>2024</year>) <article-title><italic>SciCast is on vacation</italic></article-title>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://scicast.org/">https://scicast.org/</ext-link> (Accessed: 15 May 2024).</comment></mixed-citation></ref><ref id="ref31"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Siegel</surname>, <given-names>A</given-names></string-name></person-group>. (<year>2023</year>) <article-title><italic>UK’s cosmic bazaar and US’ INFER forecasting collaboration is launched</italic></article-title>, <day>20</day><month>February</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.cultivatelabs.com/posts/uk-and-us-government-forecasting-collaboration-is-launched">https://www.cultivatelabs.com/posts/uk-and-us-government-forecasting-collaboration-is-launched</ext-link> (Accessed: 10 March 2024).</comment></mixed-citation></ref><ref id="ref32"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Simoiu</surname>, <given-names>C</given-names></string-name>., <string-name><surname>Sumanth</surname>, <given-names>C</given-names></string-name>., <string-name><surname>Mysore</surname>, <given-names>A</given-names></string-name>. and <string-name><surname>Goel</surname>, <given-names>S</given-names></string-name></person-group>. (<year>2019</year>) <article-title>‘Studying the “Wisdom of Crowds” at Scale’</article-title>, <source>Proceedings of the AAAI Conference on Human Computation and Crowdsourcing</source>, <volume>7</volume>(<issue>1</issue>), pp. <fpage>171</fpage>–<lpage>179</lpage>. doi: <pub-id pub-id-type="doi">10.1609/hcomp.v7i1.5271</pub-id>.</mixed-citation></ref><ref id="ref33"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Stastny</surname>, <given-names>B.J</given-names></string-name>. and <string-name><surname>Lehner</surname>, <given-names>P.E</given-names>.</string-name></person-group> (<year>2018</year>) <article-title>‘Comparative evaluation of the forecast accuracy of analysis reports and a prediction market’</article-title>, <source>Judgment and Decision Making</source>, <volume>13</volume>(<issue>2</issue>), pp. <fpage>202</fpage>–<lpage>211</lpage>. doi: <pub-id pub-id-type="doi">10.1017/S1930297500007105</pub-id>.</mixed-citation></ref><ref id="ref34"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Stottlemyre</surname>, <given-names>S.A</given-names>.</string-name></person-group>(<year>2015</year>) <article-title>‘HUMINT, OSINT, or something new? Defining crowdsourced intelligence’</article-title>, <source>International Journal of Intelligence and CounterIntelligence</source>, <volume>28</volume>(<issue>3</issue>), pp. <fpage>578</fpage>–<lpage>589</lpage>. doi: <pub-id pub-id-type="doi">10.1080/08850607.2015.992760</pub-id>.</mixed-citation></ref><ref id="ref35"><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Tahmasebian</surname>, <given-names>F</given-names></string-name>., <string-name><surname>Xiong</surname>, <given-names>L</given-names></string-name>., <string-name><surname>Sotoodeh</surname>, <given-names>M</given-names></string-name>. and <string-name><surname>Sunderam</surname>, <given-names>V</given-names></string-name></person-group>. (<year>2020</year>) <article-title>‘Crowdsourcing under data poisoning attacks: A comparative study’</article-title>, in <person-group person-group-type="editor"><string-name><surname>Singhal</surname>, <given-names>A.</given-names></string-name> and <string-name><surname>Vaidya</surname>, <given-names>J.</given-names></string-name> (eds.)</person-group><source>Data and applications security and privacy XXXIV</source>. vol. <volume>12122</volume>. <publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>, pp. <fpage>310</fpage>–<lpage>332</lpage>. doi: <pub-id pub-id-type="doi">10.1007/978-3-030-49669-2_18</pub-id>.</mixed-citation></ref><ref id="ref36"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Tapia</surname>, <given-names>A.H</given-names></string-name>. and <string-name><surname>LaLone</surname>, <given-names>N.J</given-names></string-name></person-group>. (<year>2014</year>) <article-title>‘Crowdsourcing investigations: Crowd participation in identifying the bomb and bomber from the Boston Marathon bombing’</article-title>, <source>International Journal of Information Systems for Crisis Response and Management (IJISCRAM)</source>, <volume>6</volume>(<issue>4</issue>), pp. <fpage>60</fpage>–<lpage>75</lpage>. doi: <pub-id pub-id-type="doi">10.4018/IJISCRAM.2014100105</pub-id>.</mixed-citation></ref><ref id="ref37"><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Tewksbury</surname>, <given-names>D</given-names></string-name></person-group>. (<year>2012</year>) <article-title>‘Crowdsourcing homeland security: The Texas virtual BorderWatch and participatory citizenship’</article-title>, <source>Surveillance &amp; Society</source>, <volume>10</volume>(<issue>3–4</issue>), pp. <fpage>249</fpage>–<lpage>262</lpage>.</mixed-citation></ref><ref id="ref38"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Tucker</surname>, <given-names>P</given-names></string-name></person-group>. (<year>2014</year>) <article-title>‘This is how America’s spies could find the next national security threat – A recent breakthrough in online prediction markets promises a better glimpse of the future – paid for by US intelligence’</article-title>, <source><italic>Defense One</italic></source>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.defenseone.com/technology/2014/02/long-overdue-return-crowdsourced-intelligence/79094/">https://www.defenseone.com/technology/2014/02/long-overdue-return-crowdsourced-intelligence/79094/</ext-link> (Accessed: 10 March 2024)</comment>.</mixed-citation></ref><ref id="ref39"><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><surname>Zegart</surname>, <given-names>A</given-names></string-name></person-group>. (<year>2021</year>) <article-title>‘Spies like us: The promise and peril of crowdsourced intelligence’</article-title>. <source><italic>Foreign Affairs</italic></source>, <day>22</day><month>June</month>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.foreignaffairs.com/reviews/review-essay/2021-06-22/spies-us">https://www.foreignaffairs.com/reviews/review-essay/2021-06-22/spies-us</ext-link> (Accessed: 15 May 2024).</comment></mixed-citation></ref></ref-list></back></article>
