Web Engineering Overview: A Historical Perspective

Copyright Jack G. Zheng, April 2006, July 2005

A version of this article also appears in "Encyclopedia of Networked and Virtual Organizations", edited by Goran Putnik and Maria Cunha, from the Information Reference Source, 2008

1. Evolvement of the web information system concept;
2. Web application development characteristics and major techniques and methodologies;
3. Development of web engineering as a discipline and as a community

Web information system

The core artifact of web engineering is the information and computer application based on the web, which was initially developed by Burners-Lee. Many terms have been used, including homepage, web page, web document (White 1996), web site (Powell et al. 1998, Lowe 1999), web(-based) application (Lowe 1999, Kappel et al. 2004), web(-based) system (Lowe 2001, Murugesan et al. 1999), web(-based) information system (Isokowits et al. 1998). This reflects a diverse understanding of web information system, especially at its different evolvement stage. To be consistent, the term ¡°web information system (WIS)¡± will be used in this article.

Many people use those term interchangeably and do not specifically define the term they chose to use. But in fact the meanings of these terms are not exactly the same, especially viewed from a historical perspective. Earlier names reflects web as a publishing medium. Before the middle 1990s, web sites were mainly static and were used mainly for information or document presentation. People felt confident that "web is a publication, not a piece of software" (Stross 1996, pp.180). The term "web document" given by White (1996) also reflects WIS's document centric feature:

"Web document (or hypertext document) is a collection of Web pages (hypertext pages). Each Web page is typically a single computer file ... A Web site can be thought of as a single Web document if it represents a single, logical information space."

During the late 1990s, with the wide spread of server side technologies such as CGI and ASP, and database connectivity technologies such as ODBC, more functionalities had been designed into the web. The scope of WIS had been expanded to include more complex applications, such as transaction, analysis, information management, entertainment, collaboration, etc. It also grew into a platform to support complex enterprise systems. The web then was eventually perceived to be software applications besides just information and documents. Thus the term such as web application and web system have been used to differentiate it from web page or web document. The definition given by Holck (2003) indeed captures this evolving role of web:

¡°(WIS is) a computer-supported information system, utilizing the technology of the WWW, and accessed by the majority of its users via a browser.¡±

Christodoulou and Papatheodorou (2005) go further to distinguish the term web page, web application and WIS, thus gives more complete understanding of these related terms:

¡°WIS is an information system utilizing Web technologies to provide information (data) and functionality (services) to end-users through a hypermedia-based presentation/interaction user interface on web-enabled devices.

Web applications are the different functionality-oriented components of a WIS. A web application is actually a small-scale WIS, providing very specific information or functionality. Many developers use these terms as synonymous, especially for small WISs.¡±

Here web technologies refer to web based programming/scripting languages, markup languages, databases, servers, protocols, application frameworks/architectures, etc.

This is a fairly good definition for WIS for its development in late 90s and earlier 2000s. However, even this definition cannot catch up with the fast growth of WWW. For example, it does not reflect the latest development of web services, which do not provide user interfaces. The definition above can be changed slightly to accommodate this new development:

Web information system is an information system that utilizes web technologies to deliver information and services, to users or other information systems/applications. Web application is a specific functionality-oriented component that utilizes web technologies to deliver information and services to users or other applications/information systems. A web information system usually consists of one or more web applications, together with information components and other non-web components.

However, even this definition is likely to be revised in the near future, as web is still growing at a fast pace. Any definition at present will soon be questionable because of new systems or new applications that expand the use of WWW. The diversity of WISs and web applications also creates problems for a good definition, because it is hard to have an accurate and complete definition to cover all aspects. This is also a difficulty for a good classification of WIS.

Classification of web information systems

There are different types of WIS and they vary widely in its purpose, scope and complexity. A classification of WIS will help to understand WIS more in details, and it is necessary to do so because different WIS potentially needs different approaches to develop and maintain. However, there is no complete or widely accepted way to categorize WIS (Murugesan and Ginige 2005). Trying to include and classify all kinds of WISs is difficult, especially it is still growing.

Earlier classifications are pretty simple and they tend to be based on the scope and complexity of WIS. Powell et al. (1998) simply categorizes web sites as internet (public access), intranet (internal to an organization) and extranet (limited access to public). Such classification was also used by Isakowitz et al. (1998), which further divides public access sites into ¡°web-presence sites that are marketing tools designed to reach consumers outside the firm¡± and ¡°electronic commerce systems that support consumer interaction, such as online shopping¡±. Powell et al. (1998) also categorizes web sites by complexity levels into static (pure static webpages), dynamic (database driven), interactive (dynamic data access) and fully functional sites (supporting processes and transactions).

Later more meaningful classification is presented as web information systems are understood more. Ginige and Murugesan (2001), Deshpande et al. (2002), Murugesan and Ginige (2005) provide similar taxonomies based on WIS functionality: informational (news, classifieds), interactive (information search, customized information presentation), transaction (shopping, banking, billing), workflow oriented (planning and scheduling, management), collaborative (group/team site, Wiki), online community (forum, BBS), and web portals (which provides a mixture of information and functionality above).

The above classifications are correspondent to the more traditional perception of WIS as websites. They do not cover new developments of WIS. Thus, the following classification is suggested to include two other types of WIS which are not covered in previous classifications:

Again, just like the definition for WIS, these classification schemes may be updated in near future. It is a challenge as well as an opportunity to establish a more complete taxonomy of WIS, potentially with hierarchies. That will lead to a much better understanding of WISs and web applications.

WIS development: an engineering approach

Many web developers do not have background of software development. It is common that ¡°the process of HTML or graphic design is often confused with the complete process of web design and publishing¡± (Powell et al. 1998, page 1). Now people do realize WIS development is not just web page creation; in many instances, it can grow very complex and go beyond the control of a single ¡°webmaster¡± (Powell et al. 1998). Good WISs are not just about artistic look and feel; it is also about quality attributes such as usability, accessibility, scalability, maintainability, compatibility and interoperability, security and reliability (Murugesan and Ginige 2005). Engineering approaches are needed to meet these quality attributes.

Characteristics of WIS development

WIS usually has multiple perspectives as software system, hypermedia system, information repository and network system. Its development process (especially that of large scale WIS) has its own unique characteristics that are quite different from traditional information systems development (Murugesan and Ginige 2005, Kappel et al. 2004, Christodoulou and Papatheodorou 2005, Powell et al. 1998, Lowe 1999, Lowe 2001, Deshpande et al. 2002):

1. The content is important. ¡°The development often contains the development of content itself¡± (Powell et al. 1998). WIS typically presents large volume of information, and it is very common that information is non-structured or semi-structured. Specific and careful content (information) design is needed for web information seeking.
2. It is very focused on presentation. Many WISs are associated with organization image and the presentation is part of a marketing strategy. There is ¡°a greater bond between art and science¡± (Powell et al. 1998). Moreover, web interface design is very different from and more complex than traditional GUI design because of the uniqueness of browser and web architecture.
3. WIS is based on the Internet technologies (such as TCP/IP) and web technologies. There is a vast body of them with rapid changes. Coordination among these technologies is often a problem. WIS is often multi-tiered and multi-layered, often comprises of distributed and heterogeneous data, applications and systems. The need for integration of applications and information is often more pronounced.
4. WIS constantly evolves. Changes are continuous and frequent. There is usually no clear line between development and maintenance. Web engineering is more about growth and evolution. That often requires the development process adapted to small changes and quick deployment.
5. There is a greater level of requirements volatility and client uncertainty, which makes web project life cycle dramatically shorter and smaller in scale.
6. WIS development consists of diverse development activities, and the team of web developers usually has very different background, skills and knowledge, which at least include graphics, database, hypermedia, programming, security, networking and all kinds of servers. It is much more difficult to manage such team and process.

1. Information Structure: web pages are organized by navigational patterns of the information space, including sequential, hierarchical (tree), grid, network (web) and the hybrid of these.
2. Relationship Management Methodology (RMM): proposed by Isakowitz et al. (1995), RMM is based on the entity relationship analysis of content. It is a structured media design that is highly useful when information is structured and volatile (such as product catalogs).
3. Information Mapping: web content is organized based on information blocks and information maps/pages that are logically related and constitute a subject (hypertrail).

1. WIS and WIS development process are different and unique, as described in the prior section of this article.
2. Web engineering is multi-disciplinary; no single discipline (such as software engineering) can provide complete theory basis, body of knowledge and practices to guide WIS development.
3. Web based information systems and applications are pervasive and non-trivial. The prospect of web as a platform will continue to grow and it is worth being treated specifically.

However, it has been controversial, especially for people in other traditional disciplines such as software engineering, to recognize web engineering as a new field. The issue is how different and independent web engineering is, compared with other disciplines.

There have been some debates on this issue and some people perceive web projects are only different in its short development cycles and time pressure to the market (Pressman 1998). Glass (2003) accept those differences but states that ¡°software project are always different¡±, ¡°why should we be surprised that the same condition holds for Web and non-Web projects?¡± Ramesh et al. (2002) also thinks that these aspects of the web development processes exist in some traditional software development, but it¡¯s the intensity with which they apply that distinguishes web development. Holck (2003) questions the differences and raise a new point that ¡°what is new is not the web development in particular, but rather the whole field of IS services, their delivery and associated organization process¡±.

The issue is also reflected in the slow adoption of web engineering in institutional programs. Except for very few institutions -- for example, University of Western Sydney has offered a master of information technology in web engineering and design program since 1999 (Deshpande 2004) -- most colleges do not offer separate curriculum and courses on web engineering. Many courses offered are still focused on very narrow aspects of web development, most often web page design or web programming. Hopefully the situation will change in the future with the proposition of web engineering curriculum models (Whitehead 2002, Hadjerrouit 2005) and a new textbook on web engineering (Kappel et al. 2006, to be published).

Trying to determine whether web engineering is a new discipline at this time is difficult; both proponents and opponents have sound reasons. Any establishment of a discipline needs time and the acceptance as a discipline is a long process. Web engineering is still young and maybe it is still early to determine it is a discipline. It still lacks core theories, core research areas and boundaries. The field of software engineering itself once had the same problem. As the field matures, it will establish its own core areas and generate its own body of knowledge.

Web engineering topics

On the other hand, web engineering is facing a problem of scope. Web engineering is a diverse field that covers extensive topics. These topics are usually related to other disciplines, such as software engineering, hypermedia engineering, human-computer interaction, network engineering, project management, web programming, databases and information management (Christodoulou and Papatheodorou 2005). However, there are no commonly and clearly defined core areas in this field, a similar situation with the field of Information Systems. Murugesan et al. (1999) identified 16 major areas; ICWE 2005 listed 31 topic areas and ICWE listed 7 categories and over 50 topics. On one hand this reflects the prosperity of web engineering research activities, but on the other hand, such diversity of topics may lead to confusion about web engineering as a discipline. What exactly does web engineering study? An emergent task is to organize these topics by some scheme so that the domain can be understood more clearly. A scheme is suggested here to organize topics by core areas, closely related areas and distant related areas (table 2).

Although this classification is preliminary and tentative, it is a good start to understand the domain systematically. The boundary of web engineering field depends on people¡¯s perception of the field whether it should be more open or more bounded. This will be a long process until the field matures.

Web engineering community

Although it has more difficulties to be recognized as a discipline, web engineering as a community has strong growth for the past 10 years with the web itself. The problems of developing and managing websites were first addressed in the World Wide Web Conferences, the earliest and largest conference serials on subjects focused on web. At the fifth World Wide Web Conference 1996 in Paris, France, Bebo White (Stanford, USA), organized a tutorial about Web Document Engineering (White 1996). At the 7th World Wide Web Conference in Brisbane, Australia in 1998, San Murugesan (University of Western Sydney, Australia) organized the first workshop on Web Engineering. The workshop continued until 2001 and became part of the main WWW conference program in 2002. The Web Engineering Track was established as an Alternate Paper Track, co-chaired by Yogesh Deshpande, Martin Gaedke, San Murugesan and Daniel Schwabe. In the next year, web engineering became a Refereed Paper Track at the twelfth WWW Conference (2003) at Budapest, Hungary. Since then web engineering has been assimilated into a number of tracks in subsequent WWW conferences to cover more topics and research outcomes.

Besides its recognition in the WWW conference, such workshops on web engineering had also been successfully held with the International Conferences on Software Engineering (ICSE) in 1999, 2000 and 2002. These events reflected a growing concern and need for a field of its own on development, operation, and evolution of web information systems and applications.

Web engineering has eventually established its own organization, journals and conferences to draw attention from researchers and practitioner in different areas. In 2001, the first International Conference on Web Engineering (ICWE) was held in C¨¤ceres, Spain. It has been held annually and the most recent one will be held at Stanford SALC in 2006. The workshop on website evolution (WSE) (http://www.websiteevolution.org) had been held annually since 1999. In 2000, the first international conference on Web Information Systems Engineering (WISE) was held in Hong Kong and continued its serials. These conferences truly reflect a global awareness of web engineering (table 3).

Besides conferences, the Journal of Web Engineering (JWE), the first international Journal related to Web Engineering, was published by Rinton Press, NJ, USA in 2002. The Journal of Web Engineering (JWE) provides a forum for distribution of information and knowledge in all areas of Web Engineering. In 1998, the domain and the website of ¡°webegineerng.org¡± was established and hosted by the University of Karlsruhe, Germany. All these events and activities show a strong growth the web engineering community.

Conclusion: opportunities and challenges

Web is young, and web engineering is young. With the fast growth of WIS, its meaning, definition, classification, development methods and techniques are likely to change in near future. We have not fully understood all emerging web related concepts and processes yet. WIS is something different; information systems development and many disciplines (such as software engineering) are changing because of web.

Although there have so much debate on whether web engineering is really different or should be treated differently, the fact is that it has achieved wide awareness and acceptance, with established conferences, books and communities all round the world. There are unprecedented opportunities and resources to push the web engineering forward.

References:

• Ceri, S., Fraternali, P., and Bongio, A. "Web Modeling Language (WebML): a Modeling Language for Designing Web Sites," 9th WWW Conference, Amsterdam, 2000.
• Chen, J.Q., and Heath, R.D. "Web Application Development Methodologies," in: Web Engineering Principles and Techniques, W. Suh (ed.), Idea Group, 2005, pp. 76-96.
• Christodoulou, S.P., and Papatheodorou, T.S. "Web Engineering Resources Portal (WEP): A Reference Model and Guide," in: Web Engineering Principles and Techniques, W. Suh (ed.), Idea Group Publishing, 2005, pp. 31-74.
• Conallen, J. Building Web Applications with UML, (Second Edition ed.) Addison-Wesley, 2003.
• Deshpande, Y. "Web Engineering Curriculum: A Case Study of an Evolving Framework," International Conference on Web Engineering, 2004.
• Deshpande, Y., Murugesan, S., Ginige, A., Hansen, S., Schwabe, D., Gaedke, M., and White, B. "Web Engineering," Journal of Web Engineering (1:1) 2002, pp 3-17.
• Gellersen, H.-W., Wicke, R., and Gaedke, M. "WebComposition: An Object-Oriented Support System for the Web Engineering Lifecycle," 6th WWW Conference, 1997.
• Ginige, A., and Murugesan, S. "Web Engineering: An Introduction," in: IEEE Multimedia, 2001, pp. 14-18.
• Glass, R.L. "A Mugwump's-Eye View of Web Work," in: Communications of the ACM, 2003, pp. 21-23.
• Graham, I. A Pattern Language for Web Usability, (1st edition ed.) Pearson Education, 2003.
• Hadjerrouit, S. "Designing a Pedagogical Model for Web Engineering Education: An Evolutionary Perspective," Journal of Information Technology Education (4) 2005.
• Holck, J. "4 Perspectives on Web Information Systems," 36th Hawaii Internation Conference on System Sciences (HICSS'03), 2003.
• Isakowitz, T., Bieber, M., and Vitali, F. "Web Information Systems," Communications of the ACM (41:7), July 1998, pp 78-80.
• Kappel, G., Michlmayr, E., Pröll, B., Reich, S., and Retschitzegger, W. "Web Engineering - Old wine in new bottles?" 4th International Conference on Web Engineering (ICWE 2004), Munich, Germany, 2004, pp. 6-12.
• Kappel, G., Pröll, B., Reich, S., and Retschitzegger, W. (eds.) Web Engineering. John Wiley & Sons, 2006.
• Lowe, D. "Web Engineering or Web Gardening?" WebNet Journal (1:1), Jan-Mar 1999.
• Lowe, D., and Henderson-Sellers, B. "Characteristics of Web Development Process," SSGRR-2001 Infrastructure for E-Business, E-Education, and E-Science., L'Aquila, Italy, 2001.
• Murugesan, S., and Ginige, A. "Web Engineering: Introduction and Perspectives," in: Web Engineering Principles and Techniques, W. Suh (ed.), Idea Group Publishing, 2005, pp. 1-30.
• Murugesan, S., Deshpande, Y., Hansen, S., and Ginige, A. "Web Engineering: A New Discipline for Web-Based System Development," First International Conference of Software Engineering (ICSE) Workshop on Web Engineering, University of Western Sydney, Australia, 1999.
• Neuwirth, C.M., and Regli, S.H. "Usability and the Web," in: IEEE Internet Computing, 2002, pp. 44-45.
• Nielsen, J. Usability Engineering Morgan Kaufmann, 1994.
• Nielsen, J. Designing Web Usability New Riders Press, 1999.
• Powell, T.A., Jones, D.L., and Cutts, D.C. Web Site Engineering - beyond Web Page Design Prentice-Hall, 1998.
• Pressman, R.S. "Can Internet Applications Be Engineered?" in: IEEE Software, 1998, pp. 104-110.
• Ramesh, B., Pries-Heje, J., and Baskerville, R. "Internet Software Engineering: A Different Class of Processes," Annals of software engineering (14:4), December 2002, pp 169-195.
• Schwabe, D., and Rossi, G. "Building Hypermedia Applications as Navigational Views of Information Models," 28th Annual Hawaii International Conference on System Sciences (HICSS), 1995.
• Stross, C., "The Web Architect's Handbook," Addison-Wesley, Harlow UK, 1996
• White, B. "Web Document Engineering (based on a tutorial presented at the World Wide Web Conference WWW5, Paris, 1995)," SLAC, May 1996.
• Whitehead, J.E. "A Proposed Curriculum for a Master in Web Engineering," Journal of Web Engineering (1:1) 2002, pp 18-22.

Web Engineering Resources