Parallel processing systems will be studied from a queueing theoretic point of view, under general assumptions of stationarity and ergodicity of the random input flow of jobs. The jobs have some complex internal structure, consisting of multiple elementary tasks, that have to be executed by a system of processors, according to some specifications. The classical queueing theoretic problems of stability, asymptotic stationarity etc. will be addressed, as well as control theoretic problems of optimal dynamic scheduling of jobs to processors in order to optimize the performance of the system, for example, maximize the throughput. Analysis will focus on basic prototypical models of parallel processing, capturing the essential operational characteristics of large classes of parallel processing systems. The research is important for understanding the operation of parallel processing systems at a fundamental mathematical level and establishing techniques to optimize their performance, based on a solid mathematical methodology, contrary to the largely heuristic current practice.