Emerging Do-It-Yourself database-driven web application projoect aims to (1) enable non-programmers to rapidly build custom data management and workflow applications and (2) to promote a novel pattern of interaction between application owners and programmers. Their beneficiaries are organizations, in need of long tail web applications, that cannot afford the time and money needed to engage into the conventional code development process.
Do-It-Yourself platforms must maximize two metrics that present an inherent trade-off: the simplicity of specification and the application scope, which characterizes the class of applications that can be built using the platform?s specification mechanism. The proposal introduces two scopes with interesting trade-off features. Namely, in the "All-SQL" scope and the (more limited) "forms-driven workflows" scope each application page consists of a report (modeled by a nested query) and forms and actions in the report's context, leading to updates. The limitations have practically minor effects on the scope but they enable simple specification and automatic optimizations, studied in the proposal, such as: 1. automatic creation of reports by choosing between the candidates using information theoretic criteria relying on constraints captured in the limited models. 2. summarization of applications as workflow specifications by analyzing the dependencies between updates and queries. Vice versa, the proposal shows how simple workflow primitives translate to queries (reports) and updates (forms and actions). The proposal also provides an unlimited model of web applications, where programmers introduce code components and interface them with the "limited" part via queries and updates.
The proposed models of database-driven web applications will impact the education of both Computer Science (CS) and non-CS students that need to comprehend web applications at a high conceptual level. For further information see the project web page at www.db.ucsd.edu/forward
Creating even a conceptually simple web application for data analysis and basic process coordination takes an inordinate amount of time and money. The goal of this project was to enable the inexpensive and rapid development of data-driven applications. In the case of the simpler applications, development should become so easy that their development should be "Do-It-Yourself (DIY)", i.e., their developer may be just a "dabbler" in Information Technology, who likely lacks formal Computer Science training. Rather programming can reduce to pluging together data and visualization components. The project's first task was to investigate why the development of (even simple) applications requires many lines of code and high expertise. The project observed that web application developers integrate many sub-systems in order to deliver even simple data-driven applications. They employ libraries of visual components (e.g. maps, calendars, tabbed windows) to achieve the user friendliness and aesthetics of modern web applications, relational databases to read/write persistent data consistently and concurrently, etc. Consequently, developers write large amounts of "plumbing" code to copy data to and from these disparate sub-systems. Investigating deeper in the question, the project classified the categories of plumbing code and found that typically code is spent on mundane, low-level integration problems that result from: • Language heterogeneities between the browser’s HTML / JavaScript, the application server’s (Java or other programming language) and the database server’s SQL, coupled with the distribution of the three computation points (browser, app server and database server). The distribution and heterogeneities require code that translates the SQL data into Java objects, then into JavaScript objects, and then vice versa. • Coordinating data dependencies with imperative code. For each user action in an application the developer has to implement code that manually updates the current page according to the dependencies between the page's data and the effected user action. • Deployment. A mundane, yet very real problem, is that the developer has to set up and operate multiple systems (app server, database server, network connection) in order to deploy his application. He has to download even more systems in order to develop his application. As a solution to the above problems the project created the FORWARD cloud-based platform for the rapid development of data-driven applications. FORWARD adopted the declarative programming language SQL as the underlying programming language, since declarative languages lead to much simpler programs and SQL has a track record of success in data management. The project's research acted on creating the appropriate system architecture, programming language design (extended SQL to SQL++) and algorithms that automate system integration and data coordination tasks that would need to be manually performed otherwise. The technical achievements included: 1. Simple applications can be developed on FORWARD without using any Java or JavaScript code, therefore resolving the "Language Heterogeneity" problem. 2. The user interface is specified by page configurations, which visualize dynamic data generated by SQL++ simply by annotating with what kind of visualization they should be created. (See image where the result of an SQL query is annotated to be presented as a map. An alternate visualization into a barchart would simply need a different visualization annotation.) The FORWARD developer need not worry about coordinating data from the server to the visualization, which resolves the "Data Coordination" problem. 3. JavaScript need to be written only if one needs to create a custom visual unit (i.e., if the repertoire of third party maps, barcharts etc is not enough for his application). 4. Java need to be written only for computations not easily expressible in SQL. Points (3) and (4) enablea continuum: While SQL with visualization annotations guarantees that simple things are done easily, Points (3) and (4) guarantee that non-simple extensions can also be made to an application. A central contribution of FORWARD is the provision of an SQL++ accessible unified application state virtual database that, besides the persistent database of the application, includes transient memory-based data, notably the page’s data, which in conventional web application programming would require Java and Javascript to be accessed. Furthermore, FORWARD guarantees that the browser’s user-collected data are correctly reflected into the unified application state’s page data before they are used by the programs. The automatic reflection resolves a critical "Data Coordination" problem. The project embodied its research on the cloud-based FORWARD application platform, available at forward.ucsd.edu. One can develop and deploy programs directly at forward.ucsd.edu without the need to download any software on his/her computer or the need to set up a web accessible application server. For data hosting, one may use the cloud-based database of FORWARD or may have FORWARD remotely connect to his/her own database. Finally, the project transfered inthe classroom its lessons on how to rapidly and cleanly build data-driven web applications, with or without the benefit of FORWARD.
