There has been a recent proliferation of industrial-strength synchronous collaborative systems, however, there has been little research comparing the performance of their distributed architectures, even though inadequate performance can be a show-stopper. In systematically examining this architectural design space, this project will provide a critical foundation for what is currently the ''black art'' of collaborative system design. This will have broad ranging impact in both research and industry.
The architecture design space will be modeled by several architecture parameters (e.g., transport protocol, server configuration, I/O scheduling), system parameters (e.g., network latency, processing power) and task parameters (e.g., I/O size, think time) that impact performance. Equations will be developed that give the value of the time, along key metrics such as join, leave, response, feedthrough, and task-completion, as a function of the architecture, system, and task parameters. Experiments will be performed to validate these equations using logs of different kinds of collaborations such as multi-user presentations, chats, and scientific visualizations. This novel experimental test bed will enable performance comparisons that have not been possible before.
Broader impact: The analytic model resulting from this research will allow application and infrastructure developers to determine the range of architectures they should implement. It will provide scientists, engineers, writers, and other users of collaboration technology a better understanding of the design space so they can choose the correct architecture to fit their needs while appreciating the performance consequences of that choice. In addition, the project infrastructure will form the first proof-of-concept system supporting the entire space of collaborative architectures, a novel research test-bed for experimentation, and a vehicle for teaching collaboration architectures. The multi-user activity logs created for these experiments will be publicized so that they can become benchmarks, in both academic and industrial research, used to compare performance of new architectures.
The increasingly popular multi-user webinars, games, and document editing are examples of distributed synchronous collaboration technology. The interactivity of a synchronous application can be the difference between life and death in synchronous collaboration. If a system does not display the results of an action to a user in a timely fashion, the user may get frustrated and switch to a different system or, worse, never use a collaborative system again. One way to improve performance is to use faster computers and communication links. As this solution is expensive and not always possible, our research has developed techniques that attempt to automatically optimize the use of existing resources. Rather than focusing on a particular implementation, or class of applications, we have taken an application-independent approach. In general, the computers of all users interacting with a collaborative application are involved in executing the application. Each of these computers gathers input from and displays output to the local user, and communicates information to other computers. Optionally, it can also perform some of the processing of user actions (such as determining the next slide to present in a webinar), and route information to other computers. Our experiments found that the performance of an application depends on three architectural dimensions including: (1) which computers process actions, (2) how information is routed among them, and (3) how processing and routing tasks are scheduled on different computers and how many cores are available to perform these tasks. One particularly interesting result had to with a new scheduling policy we invented, which was based on psychological experiments showing that users cannot perceive delays of less than 50 milliseconds. We found that, under this policy, a non-perceptible increase in the response time of a user could result in much larger, perceptible, reduction in response times of other users. This policy is an interesting example of merging computer science and psychology research The architecture factors we identified can be used to manually tune the performance of an application, but this "trial and error" approach is tedious and error prone. Therefore, we developed mathematical equations that can predict the effect of changes to these factors, and verified that these equations were consistent with our experimental observations. The final step was to develop a self optimizing software system that (a) is independent of any particular application, (b) keeps track of various performance parameters such how much time is taken by each computer to process actions and communicate them, how much time users take to think between actions, and network latencies to all other computers, (c) automatically changes the architecture along one of the three dimensions based on predictions of our equations, and (d) keeps the overhead of monitoring parameters and changing architecture low. To illustrate a simple use of this system, suppose a user on a tablet is giving a PPT presentation to a user on a smart phone. As the smart phone is slow, all PPT processing is done on the tablet. Suppose a new user with a powerful laptop joins the session. The PPT presentation application would automatically move to the laptop when the latecomer joins (if the equations predict that this change would be beneficial), and move back to the tablet if he/she leaves. Our work is even more important to support large scalable collaborative sessions, such as state of the union webinar presentation to the whole country or the whole world. The extent to which our work can help, of course, depends on various factors such as the configuration of computers and communication links and the application, but our results show that it cannot hurt and, in many scenarios, it can help in a significant manner. Perhaps more important, it has led to a better understanding of the architectures of collaborative applications, and has led to computer software that we intend to use for teaching. Our work can be considered a natural progression of research in computer science. In the areas of distributed and real-time computing, there has been extensive work on dynamically adjusting the parameters of a system while it executes to improve performance. Ours is the first such work to consider collaborative systems. Such work is necessary as collaborative systems, by definition, involve humans, and human-perception studies have shown that we are sensitive to slight performance degradations (and fortunately, improvements). It has application in the large number of fields such as science, business, distance education, and engineering that use webinars and other synchronous collaborative systems for distributed interaction.
