ARTÍCULO ORIGINAL

 

Measuring Software Quality in Open Source Communities Through the Lens of Social Capital

 

Medición de Calidad de Software en las Comunidades Open Source través de la lente del Capital Social

 

 

Steve Whyte^1*, Maurice McNaughton¹, Delroy Chevers¹, Michelle McLeod¹

¹Mona School of Business – University of the West Indies, Jamaica

*Autor para la correspondencia: [e-mail]

 

 

---

ABSTRACT

In this paper we propose a model for assessing free and open source software (FOSS) product quality by examining the social relations that exist within FOSS communities and the extent to which the social network structures and/or the quality of trust relationships amongst key members within these communities’ impact product quality. Empirical studies suggest an increase in the adoption of FOSS products both for personal use as well as in mission critical IT systems in organizations. Consequently, as individuals and firms consider adopting FOSS solutions they are faced with the challenge of evaluating the uncertainties of key software quality related facets and this challenge is further compounded by the findings of empirical studies that argue that FOSS product quality is difficult to determine using established traditional quality models. Additionally, empirical studies argue that the centrality of members who report software bugs influences the extent to which bugs are resolved which by extension impacts on product quality. Using the constructs and measures associated with the structural and relational dimensions of social capital theory, this paper proposes a theoretical model to explore the social interactions between open source project members by examining both the social network structures as well as the quality of member relationships, using appropriate social network measures. The model also examines the extent to which these relationships are moderated by the average weighted centrality of members who report bugs in these communities.

Key words: Social Capital, Software Quality, Free and Open Source Software, Social Network Analysis

---

RESUMEN

En este artículo se propone un modelo para la evaluación de software libre y de código abierto (FOSS) la calidad del producto mediante el examen de las relaciones sociales que existen dentro de las comunidades de software libre y el grado en que las estructuras de la red social y / o la calidad de las relaciones de confianza entre los miembros clave dentro esta calidad comunidades producto impacto. Los estudios empíricos sugieren un aumento en la adopción de los productos de software libre, tanto para uso personal, así como en sistemas de misión críticos de TI en las organizaciones. En consecuencia, como individuos y las empresas consideren la adopción de soluciones de software libre que se enfrentan con el reto de evaluar las incertidumbres de los aspectos clave relacionados con la calidad de software y de este desafío se agrava aún más por los resultados de estudios empíricos que sostienen que la calidad del producto de software libre es difícil determinar utilizando los modelos tradicionales de calidad establecidos. Además, los estudios empíricos argumentan que la centralidad de los miembros que reportan los errores de software influye en el grado en que se resuelven los errores que, por impactos de extensión sobre la calidad del producto. El uso de las construcciones y las medidas relacionadas con las dimensiones estructurales y relacionales de la teoría del capital social, este documento propone un modelo teórico para explorar las interacciones sociales entre los miembros del proyecto de código abierto mediante el examen tanto de las estructuras de las redes sociales, así como la calidad de las relaciones miembro, utilizando medidas de redes sociales apropiadas. El modelo también examina el grado en que estas relaciones son moderadas por la centralidad promedio ponderado de los miembros que informan de errores en estas comunidades.

Palabras clave: capital social, calidad del software, software libre y de código abierto, Análisis de Redes Sociales

---

 

 

INTRODUCCIÓN

With the widespread penetration and use of the internet globally (Adewumi, Misra, & Omoregbe, 2013), there has emerged a phenomena in which distributed virtual communities collaborate to create scalable and reusable computer software free of charge, known as open source software, for anyone who wishes to use and/or modify based on individual needs  (von Krogh, Haefliger, Spaeth, & Wallin, 2012). These open source software groups are generally comprised of individuals and organizations who participate on a voluntary basis and with no direct financial expectations (von Krogh, Haefliger, Spaeth, & Wallin, 2012).

Users of FOSS solutions are faced with the challenge of the uncertainties of determining key software quality related facets such as development and product continuity (Kaur & Singh, 2015), product selection, documentation, community support, maintenance, legal, migration as well as issues related to usage (Stol & Babar, 2010).  This challenge is further compounded by the fact that FOSS product quality is difficult to determine using established traditional quality models (Adewumi, Misra, & Omoregbe, 2013). The seminal literature outlining Raymond's proposition on FOSS quality is that "given enough eyeballs, all bugs are shallow", meaning, the more widely available the source code is for public testing, scrutiny, and experimentation, the more rapidly all forms of bugs will be discovered (Raymond E. , 1999). The argument is that quality is therefore assured since everybody can access, review and improve anyone’s work (Lussier, 2004).  There is conflict in the literature however as empirical studies have shown that quality is not guaranteed just by the mere fact that the source code is available for public viewing and testing. For example, Aberdour (Aberdour, 2007) argued that compared to the proprietary software environment, open source projects have a higher potential to develop faster and improve its quality because more people can access them and these software are peer reviewed by “unbiased” people having no vested interest in such projects. In contrast however Bouktif et al. (Bouktif, Antoniol, Merlo, & Neteler, 2006) argue that the open source software phenomenon suffers from frequent changes, increase in complexity and quality deterioration. Ruiz & Robinson (Ruiz & Robinson, 2011) in conducting a review of the FOSS literature to understand the reason for the level of conflict in the literature on open source software quality conclude: 
“….there is little consensus in the FLOSS literature when it comes to defining quality. With this literature review, we found the reason for these mixed results is that quality is being defined, measured, and evaluated differently. We report the most popular definitions, such as software structure measures, process measures, such as defect fixing, and maturity assessment models. The way researchers have built their samples has also contributed to the mixed results with different project properties being considered and ignored. Because FLOSS projects are evolving, their quality is too, and it must be measured using metrics that take into account its community’s commitment to quality rather than just its software structure. Challenges exist in defining what constitutes a defect or bug, and the role of modularity in affecting FLOSS quality”. (Ruiz & Robinson, 2011)

At a United Nations Economic Commission for Europe (UNECE) conference of European Statisticians held in Dublin, Ireland on 14th April 2014, a working paper which was the result of a study to explore open source software benefits for the statistics industry was presented by Brian Buffett of the UNESCO Institute for statisticians. One of the key objectives of the study was to review the current literature on the benefits and challenges affecting the adoption of open source software in the statistics industry. Another critical objective was to explore and measure by way of survey whether organizations consider open source software as a means of achieving business goals and in which aspects of the industry is open source software being utilized in their operations. Survey data was collected between 2011 and 2013 from chief statisticians and participants from both national and international public sector statistical organizations with most respondents representing organizations such as national statistical institutions, central banks and some public sector international organizations based in Europe and North America. A summary of both the results from the literature review as well as from the survey reveal interesting similarities and the results are tabled as follows: Table 1

 

LITERATURE REVIEW

In the following sub-sections we will review the existing literature on traditional software quality measures, software quality and the FOSS environment, FOSS maturity models, social capital theory, team performance in FOSS communities as well as online forums in FOSS communities which will provide the context for the next section as well as the basis for further research which is the proposed model for measuring FOSS quality.

Traditional Software Quality Measures

From the seminal literature the origins of software quality can be traced back to industrial engineering and operations management. For these fields of study, quality is defined as adherence to process specification (Deming, 1982) or the creation of a product that meets customer requirements with zero defects (Crosby, 1979). The word quality has its origins from the Latin word “quails” meaning “such as the thing really is”. From the seminal literature Dale et.al (Dale & Bunney, 1999) describe quality as the “totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs”. The Institute of Electrical and Electronics Engineers (IEEE) defines quality as, “the degree to which a system, component or process meets specified requirements” or “the degree to which a system, component or process meets customer or user needs or expectations” (Adewumi, Misra, & Omoregbe, 2013).  Software quality is therefore evaluated based on a combination of several factors which must ultimately be measured by appropriate metrics (Vanitha & ThirumalaiSelvi, 2014). In general, the seminal literature on quality positions quality within the context of two perspectives:

• Perspective (1) - Fitness for purpose. This approach imputes high quality to a product once it is fit for its purpose and does what it is supposed to do (Coleman & Manns, 1996). In order to measure fitness for purpose one has to measure the product deliverables against a pre-established specification. Unfortunately, this approach, known as conformance to specification, raises one key issue in that it assumes that the specification itself is of a high quality.

• Perspective (2) - Quality attributes. As posited by Wong (Wong, 2006), quality is evaluated based on the combination of attributes that provide the greatest satisfaction to a specified consumer. The Boehm quality model is known for using this approach (Boehm, Brown, & Lipow, 1976).

Other models such as the ISO/IEC 25010: 2011 software quality model provides a list of characteristics of quality for the evaluating process. These attributes are: Functionality, Reliability, Usability, Efficiency, Maintainability and Portability.  Table 2

Software Quality and the Foss Environment

Tosi & Tahir (Tosi & Tahir, 2013) conducted an empirical survey on a set of 33 well known FOSS projects to understand how developers performed quality assurance activities for their FOSS projects. During the development lifecycle the main goal of testing is to detect software bugs (Tosi & Tahir, 2013). FOSS, as compared to closed source software (CSS) however is different with regard to development. These differences restrict the applicability of the well-established testing approaches developed for CSS in the open source domain (Tosi & Tahir, 2013). For their test results Tosi & Tahir (Tosi & Tahir, 2013) concluded that:

• Only 58% of the projects have either a test suite or some form of testing activity.

• The larger the project size, the smaller the time spent to test.

• Only 15% of the projects created testing plans.

• Most testing was done at the unit level rather than at the integration or system levels.

• Only 24% of the projects had proper testing documentation which includes test specifications, test design, test procedures, test plans and test results.

• 36% provides some (often preliminary) information on the test strategy and the test approach

• 42% has testing documentation (often incomplete and out-of-date).

• 18% exploits available testing tools but none of the projects uses a testing framework to support the whole testing process.

These findings are in line with another similar research conducted by L. Zhao and S. Elbaum (Zaho & Elbaum, 2003) which confirm that OSS is usually not validated enough and therefore its quality is not revealed enough. Stol & Babar (Stol & Babar, 2010) conducted a review of the literature on understanding the potential challenges surrounding the use of Open Source Software (OSS) components in product development.

They identified the reported benefits as outlined in the literature of using OSS components such as lower purchasing costs, availability of high quality products, adherence to open standards and no vender dependency (Stol & Babar, 2010).  Yet despite these benefits, the literature has also highlighted the challenges surrounding the use of OSS solutions including uncertainty of the quality of any particular FOSS component or product due to the sheer enormity of choices available at any single repository, in short, too much choice. Quality in this context was typically referred to in terms of quality attributes such as usability, reliability and performance. Other challenges included the lack of time to evaluate components, deciding which fork to follow as the project evolves –a fork is a spinoff from the original project direction and this usually occurs whenever core developers have disagreements about the future of the project usually resulting in delays-, lack of maintaining proper documentation, community dependence for further updates and upgrades, integration, migration and challenges relating to maintaining custom changes. It should be noted that the researches did identify possible solutions for each of the challenges identified, some of these solutions actually coming out of actual literature that identified the challenges (Stol & Babar, 2010).

FOSS Maturity Models

Open source maturity models emerged due to the inability of traditional quality models to measure unique features of open source software. According to Navica (2012), “The Open Source Maturity Model is a vital tool for planning open source success”. Some of the more popular FOSS quality models are listed below: Table 3

In addition to evaluating a FOSS product’s maturity, an Open Source Maturity Model’s (OSMM) also considers a comparison of the software with commercial alternatives as well as its correspondence to specific business but especially IT requirements. The Maturity Model therefore provides a guideline how a FOSS product should be assessed. Some of the limitations identified in some of the above maturity models as identified by Haaland et al (Haaland K. , Groven, Glott, & Tannenberg, 2010) include:

• Some of the quality models reduce the notion of quality to a few relatively simple and static attributes which are narrow in perspective.

• Some of the attributes are subjective in nature with limited scope (ex. values ranging between 1 and 5 with 1 meaning STRONGLY DISAGREE and 5 meaning STRONGLY AGREE.

• Some of the models (ex. OpenBRR) is based solely on the skill of the evaluators.

• The difference between the categories and number of quality metrics and attributes are significant across the various quality models.

• Some of the models tend to be manual and descriptive rather than automated and analytic.

• Some of the models require the direct involvement of expertise knowledge in the interpretation of the assessment results and this approach makes the models vulnerable to subjective biases and that the relatively limited number of metrics that can be examined increases the likelihood of missing important quality issues.

Future FLOSS maturity models therefore call either for an even stronger reliance on tools support, whereby the predetermination of the results as being good, bad, or neutral must be minimized; or for an integration of the human factor and further efforts to minimize the subjectivity that is incorporated by doing so. This requires an active community working with the models and legitimizing it.

Social Capital Theory

The main argument of social capital is that networks of relationships constitute a valuable resource in the conduct of specific social affairs and grants their participants with collectively-owned capital, a credential which entitles them to credit, in the various sense of the word (Szeman & Kaposy, 2010). Bourdieu’s (Bourdieu & Wacquant, 1992) definition of social capital being the resources that result from social structure is often cited whenever there is a discussion on social capital. The Organization for Economic Co-operation and Development (OECD) defines social capital as "networks together with shared norms, values and understandings that facilitate co-operation within or among groups" (Cote & Healy, 2001). The World Bank is more expansive and suggests: "Social capital refers to the institutions, relationships, and norms that shape the quality and quantity of a society's social interactions. Social capital is not just the sum of the institutions which underpin a society – it is the glue that holds them together."One of the most well-known theorists within the social capital paradigm is Putnam. He along with Woolcock have defined social capital by focusing on social networks and while their predecessors consider social capital an attribute of the individual, Putman developed it as an attribute of communities. Social Capital Theory provides a collective context in which individual relationships are embedded within a network of relationships (Granovetter, 1985). Social capital encapsulates the network as well as the resources that may be mobilized through the network (Bordieu, 1986). Social capital can be applied to both the individual or group levels (Hinds, 2008). Five key dimensions have been identified as useful proxies for social capital: Groups and networks - collections of individuals that promote and protect personal relationships which improve welfare; Trust and Solidarity – elements of interpersonal behavior which fosters greater cohesion and more robust collective action; Collective Action and Cooperation - ability of people to work together toward resolving communal issues;  Social Cohesion and Inclusion -   mitigates the risk of conflict and promotes equitable access to benefits of development by enhancing participation of the marginalized; and Information and Communication - breaks down negative social capital and also enables positive social capital by improving access to information (The World Bank Group, 2011). The effectiveness with which groups and networks fulfill their roles depends on many aspects of these groups, reflecting their structure, membership and the way they function. Key characteristics of formal groups that need to be measured include: density of membership, diversity of membership, extent of democratic functioning, extent of connections to other groups.

Academic researchers Acquaah et al. (Acquaah, Amoako-Gyampah, & Nyathi, 2014) conducted an extensive review of the academic literature on the importance of social capital in mobilizing resources for the creation of value for individuals, companies and communities. In conducting the review the researchers examined 314 articles (the majority of which were associated with the disciplines of business and economics) published from 1990 to 2013 in academic and practitioner journals as well as other sources such as the World Bank. They argue from the review that there is general consensus in the literature that social capital can be classified into three dimensions – structural, relational and cognitive. Structural social capital refers to the associated links and networks which can be objectively verified either by observation or historical records (Harpham, 2008). Structural social capital therefore refers to the structure or pattern of connections between actors by examining who are the individuals you reach, how you reach them, and how frequently you share resources and information (Nahapiet & Ghoshal, 1998). Relational social capital focuses on the quality of the interactions and the resources that are created or leveraged through the relationships. Its attributes include trust, trustworthiness, respect and friendship (Nahapiet & Ghoshal, 1998), while cognitive social capital refers to “what people feel (values and perceptions)” (Harpham, 2008) as it represents resources obtained from a common set of goals, a shared vision, and shared representations.

The social networks and ties embedded in the structural and relational dimensions of social capital have been further classified based on the strength and diversity of the ties (bonding, bridging and linking), the direction of the ties (horizontal and vertical) and the formality of the ties (formal and informal). Bonding social capital refers to horizontal, tightly cohesive ties between individuals or groups sharing similar demographic characteristics. It is characterized by homogeneous networks, which tend to be inward-looking (Acquaah, Amoako-Gyampah, & Nyathi, 2014). Bridging social capital, on the other hand, refers to ties that cut across different individuals and communities (Acquaah, Amoako-Gyampah, & Nyathi, 2014). This type of social capital is based on heterogeneous and outward-looking connections with individuals from different social groups (Ferlander, 2007).

The structural dimensions that have been examined include network structural characteristics (e.g. network links, network centrality, network density, network diversity, network size, network frequency, network redundancy, institutional network, etc.); network ties (strong ties, weak ties, government officials ties, tie strength, bonding ties, bridging ties, linking ties, structural holes, etc.); association membership and institutional links; and trust (Acquaah, Amoako-Gyampah, & Nyathi, 2014). The relational dimension of social capital has been examined by measures that focus on social networking relationships and trust (Acquaah, Amoako-Gyampah, & Nyathi, 2014). The indicators used to measure cognitive social capital are mostly attitudinal and value-based and include shared norms, values and obligations; reciprocity; shared goals and mission; and attitudes and beliefs (Acquaah, Amoako-Gyampah, & Nyathi, 2014). Figure 1

Team Performance in FOSS Communities

Crowston et al. (Crowston, Annabi, & Howison, 2003) in their review of the FOSS literature identified a range of measures that could be used to assess project success. Their research identified that the most commonly cited model for traditional information system success is the one developed by DeLone and McLean (DeLone & McLean, 2003). Crowston et.al (Crowston, Annabi, & Howison, 2003) argue that the literature suggests that many of the traditional measures based on the DeLone and McLean model are either inapplicable or difficult to apply to FOSS projects based on the uniqueness of the FOSS development environment. With the challenges faced in measuring quality using traditional measures the authors from their review of the literature in considering FOSS project success as a proxy for software quality suggested alternative measures which they categorized into two groups: (1) Measures of the Output of Systems Development –which includes project stage development lifecycle and developer satisfaction- and (2) Measures of the Process of Systems Development –which includes the number of developers and level of activity-.  The literature therefore has established that in the absence of an established FOSS quality measure, project community success with its suggested measures is a suitable proxy and which will be applied to the model in this study.

Online Forums in FOSS Communities

Online forums are key to the success of open source project development and as they continue to increase in popularity (Kanuka & Anderson, 2007) it continues to provide a platform for participants to share their knowledge, expertise and experience. There are many extrinsic and intrinsic factors that motivate members to contribute to online forums and it has been empirically demonstrated that the quality of reported bugs posted in online forums are directly related to the centrality of the bug reporter in the project network and this has a direct impact on how speedily bugs are assigned and resolved (Zanetti, Scholtes, Tessone, & Schweitzer, 2013).

Proposed software quality assessment model

As outlined earlier in the literature on Social Capital Theory, three dimensions were identified namely: structural, relational and cognitive (Acquaah, Amoako-Gyampah, & Nyathi, 2014). The cognitive dimension relates to attitudes and beliefs, shared goals, mission and norms. This dimension however will not be included in the model because FOSS communities have very strong and consistent shared norms and values that give these communities their distinct characteristics. Some of these shared norms include: (1) Open Exchange - A free exchange of ideas creating an environment where people are allowed to learn and use existing information toward creating new ideas; (2) Collaborative participation – the ability to freely collaborate to solve problems; (3) Rapid prototyping – that lead to better solution found at a faster rate; (4) Meritocracy – where the best ideas win and everyone has access to the same information and successful work determines which projects rise and gather effort from the community; (5) Community – where participants bring together diverse ideas and share work facilitating creativity beyond the capabilities of any one individual (Rao, 2015). It multiplies effort and shares the work. In their review of the literature of social capital theory Acquaah et al. (Acquaah, Amoako-Gyampah, & Nyathi, 2014) argue that the literature suggests that the value of social capital has been assessed based on its potential impact on individuals, organizations, communities, nations and regions. Social capital shares some similarities with other forms of capital such as human and physical capital, in that social capital has the ability to generate external benefits that persist (Agenor & Dinh, 2013). Specifically, “these externalities and benefits include information sharing among individuals and firms, and the matching of people to economic opportunities, mutual aid and insurance, which may affect expectations and individual behavior, as well as effective collective action” (Agenor & Dinh, 2013). Acquaah et al. (Acquaah, Amoako-Gyampah, & Nyathi, 2014) proposed an integrated model that looked at the relationships between the indicators and outcomes of social capital. The model connects the indicators, outputs and value of social capital. To gain a good understanding of the value of social capital, it is necessary to distinguish between outputs and the outcomes of social capital. Outputs such as gaining access to knowledge from an organization’s members as a result of network relationships are important, but these eventually need to be translated into outcomes such as improved financial performance, increased market share, and innovation. Social capital value includes the value that it provides to companies, to individuals, to other companies and to communities (Acquaah, Amoako-Gyampah, & Nyathi, 2014).Figure 1

There are three social network structure and one relationships constructs included in the model based on the literature: closure, bridging, leader centrality and relational trust. As outlined earlier in the literature and illustrated in the model above, the antecedents of closure, bridging and relational trust as used in this research are rooted in the assertions of social capital theory in general, and in particular with regard to team and work group outcomes, while leader centrality structure refers to prior social network studies regarding team leaders and the effect of their network position on the group effectiveness of the team.

As indicated earlier in the literature several success factors have been used as proxies to give an indication of the level of quality in FOSS solutions. These proxies include the number of downloads, number and frequency of major releases, number of forks, as well as the number of reported bugs (Howison & Crowston, 2004). While success and quality are certainly not the same (Crowston, Howison, & Annabi, 2006), success however is used as an indicator of quality considering the absence of measurable quality attributes as outlined in traditional quality models (Amrollahi, Khansari, & Manian, 2014). In this regard within the context of the research model, community success is conceptualized as consisting of two dimensions: output and activity with the output dimension referring to the quantity of software that is produced by the project community while the activity dimension reflects the quantity of participation by community members. By modeling these two dimensions as having a reciprocal relationship, this suggests that the production of more software will generally lead to greater community participation, and that increased participation will tend to attract and motivate even more developers to produce more software. To the extent that higher quality software will tend to generate a greater level of community activity than lower quality software, it is suggested that community activity can also be viewed as a proxy for software product quality (Hinds, 2008).

 

CONTRIBUTIONS AND FUTURE RESEARCH

The model outlined in the previous section will be empirically tested in the next stage of the research. The purpose of this paper was to use the literature to identify the challenges in determining FOSS product quality and to use the literature to propose a model grounded in the theory of social capital and that uses appropriate measures to evaluate the structural and relational dimensions of social capital and its relationship to FOSS product quality.

 

BIBLIOGRAPHY

Aberdour, M. (2007). Achieving quality in open-source software. Software, IEEE, 24(1), 58-64.

Acquaah, M., Amoako-Gyampah, K., & Nyathi, N. Q. (2014). measuring and valuing social capital: A Systemic Review. Network for Business Sustainability South Africa. Retrieved from http://nbs.net/knowledge

Adewumi, A., Misra, S., & Omoregbe, N. (2013). A Review of Models for Evaluating Quality in Open Source Software. IERI Procedia, 4, 88-92.

Agenor, P., & Dinh, H. (2013). Social capital, product imitation and growth with learning externalities. Washington, D.C.: The World Bank Development Economics Operations and Strategy Unit.

Amrollahi , A., Khansari, M., & Manian , A. (2014). How Open Source Software Succeeds? A Review of Research on Success of Open Source Software. International Journam for Information and Communication Technology Research, 6(2), 67-77.

Amrollahi, A., Khansari, M., & Manian, A. (2014). How Open Source Software Succeeds? A Review of Research on Success of Open Source Software. International Journal of Information & Communication Technology Research, 6(2), 67-77.

Boehm, B. W., Brown, J. R., & Lipow, M. (1976). Quantitative evaluation of software quality. Proceedings of the 2nd International Conference on Software Engineering.

Bordieu, P. (1986). The forms of capital. (J. Richardson, Ed.) Handbook of theory and research for the sociology of education, 241-258.

Bouktif, S., Antoniol, G., Merlo, E., & Neteler, M. (2006). A feedback based quality assessment to support open source software evolution: the GRASS case study. 22nd IEEE International Conference on Software Maintenance, (pp. 155-165).

Bourdieu, P., & Wacquant, L. J. (1992). An Invitation to Reflexive Sociology. Chicago, IL.: University of Chicago Press.

Coleman, M., & Manns, T. (1996). Software Quality Assurance. London: Macmillan.

Cote, S., & Healy, T. (2001). The Well-being of Nations. The role of human and social capital. Organisation for Economic Co-operation and Development,. Paris.

Crosby, P. B. (1979). Quality is Free. New York: McGraw-Hill.

Crowston, K., Annabi, H., & Howison, J. (2003). Defining Open Source Software Project Success. The School of Information Studies Faculty Scholarship.

Dale, B., & Bunney, H. (1999). Total Quality Management Blueprint. Oxford, UK: Blackwell Business.

DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. 19(4), 9-30.

Deming, W. E. (1982). Quality, productivity and competitive position. MIT Center for Advanced Engineering Study. Cambridge,MA.

Ferlander, S. (2007). The importance of different forms of social capital for health. Acta Sociologica, 50(2), 115-128.

Granovetter, M. (1985). Economic action and social structure: the problem of embeddedness. American Journal of Sociology, 91(3), 481-510.

Haaland, K., Groven, A.-K., Glott, R., & Tannenberg, A. (2010). Free/Libre Open Source Quality Models - a comparison between two approaches. EUX2010sec project, “Security infrastructure for the open source EUX2010 VoIP system”, funded by The Research Council of Norway.

Harpham, T. (2008). The measurement of community social capital. In I. Kawachi,, S. V. Subramanian, & D. Kim, Social Capital and Health (pp. 51-62). New York: Springer.

Hinds, D. (2008). Social Network Structure as a Critical Success Condition for Open Source Software Project Communities. Retrieved from Cite Seer X: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.470.2955&rep=rep1&type=pdf

Howison, J., & Crowston, K. (2004). The perils and pitfalls of mining SourceForge. Proceedings of the International Workshop on Mining Software Repositories (MSR 2004), (pp. 7-11). Edinburgh, UK.
IEEE. (1998). IEEE standard for software test documentation. IEEE.

Kaur, M., & Singh, H. (2015). Bug Tracking and Analysis in Open Source Software. International Journal of Advance Foundation And Research In Science & Engineering, 1, 1-10.

Lussier, S. (2004). ‘New tricks: how open source changed the way my team works’. IEEE Software, 21(1), 68-72.

Nahapiet, J., & Ghoshal, S. (1998). Social capital, intellectual capital and the organisational advantage. Academy of Management Review, 23, 242-266.

Rao, M. N. (2015). Fundamentas of Open Source Software. Dehli: Asoke K. Ghosh, PHI Learning Privale Ltd.

Raymond, E. (1999). The cathedral and the bazaar. Knowledge, Technology & Policy, 12(3), 23-49.

Ruiz, V., & Robinson, W. (2011). Towards a Unified Definition of Open Source Quality. 7th IFIP WG 2.13 International Conference, OSS 2011 (pp. 17-33). Salvador, Brazil: Springer.

Stol, K.-J., & Babar, M. (2010, May). Challenges in Using Open Source Software in Product Development: A Review of the Literature. Retrieved June 26, 2015, from ResearchGate: http://www.researchgate.net/publication/234755377

Szeman, I., & Kaposy, T. (2010). Cultural Theory: An anthology. USA: Wiley.

The World Bank Group. (2011). Measuring Social Capital. Retrieved from Social Capital: http://go.worldbank.org/A77F30UIX0

Tosi, D., & Tahir, A. (2013). A Survey on How well-known Open Source Software Projects are Tested. ICSOFT, 42-57.

Vanitha, N., & ThirumalaiSelvi, R. (2014). A Report on the Analysis of Metrics and Measures on Software Quality Factors – A Literature Study. International Journal of Computer Science and Information Technologies, 5(5), 6591-6595.

von Krogh, G., Haefliger, S., Spaeth, S., & Wallin, M. W. (2012). CARROTS AND RAINBOWS: MOTIVATION AND SOCIAL PRACTICE IN OPEN SOURCE SOFTWARE DEVELOPMENT. MIS Quarterly, 36(2).

Wong, B. (2006). Different Views of Software Quality. In E. Duggan, & J. Reichgelt, Measuring Information Systems Delivery Quality (pp. 55-88). Hershey, USA: Idea Group Publishing.

Zaho, L., & Elbaum, S. (2003). Quality assurance under the open source development model. International Journal of Systems and Software, 66(1), 65-75.

 

 

Recibido: 01/08/2016
Aceptado: 06/11/2016