Academic year 2015-16

Real Time Audio Processing

Degree:	Code:	Type:
Bachelor's Degree in Computer Science	21492	Optional subject
Bachelor's Degree in Telematics Engineering	21768	Optional subject
Bachelor's Degree in Audiovisual Systems Engineering	21623	Compulsory subject, 4th year

 

ECTS credits:

4

Workload:

100 hours

Trimester:

1st

 

Department:	Dept. of Information and Communication Technologies
Coordinator:	Sergi Jordà
Teaching staff:	Sergi Jordà, Angel Faraldo, Daniel Gómez
Language:	English (lectures and material)
Timetable:
Building:	Communication campus - Poblenou

 

Introduction

The course “Real Time Audio Processing” is compulsory in the 4th course of the degree in Engineering of Audiovisual Systems. This course can be understood as the continuation of “Signals and Systems” (a course common to the three degrees, lectured during the two first quarters of the second course) and of “Sound and Music Processing”, a 3rd year course of the degree in Engineering of Audiovisual Systems.

The course in relation to the postgraduates and masters of the department

This course constitutes also an excellent introduction for vary courses of the Sound and Music Computing master (SMC), since it covers concepts, tools and technologies, that in the mentioned *màsters consider known, or see a lot quickly, and that up to now had fit in the previous courses of the degree of engineering.

Real Time Audio Processing (4th course AV) ->

• Real Time Interaction (master SMC)
• Audio and Music Processing (master SMC)
• Audio and Music Analysis (master SMC)

Learning Objectives

Specific and differential characteristics of both real-time audio generation, processing and interaction. Tools and programming languages suitable for these purposes.

 

Prerequisites

Signals and Systems (8, COM, 2nd course)

Complex numbers. Fourier Analysis. Discreet and continuous Fourier Transfor. Algorithm of the Fast Fourier Transform. Sampling. Linear systems. Filters.

Sound and Music Processing (AV, 3rd course)

Analysis, transformation and digital synthesis of sound. Temporal and frequential representations of musical signals. Short time FFT. Spectral modeling of musical signals. Musical applications based in audio processing.

 

 

Associated competences

Transversal Competences

Specific Competences

Instrumental 1. Capacity of analysis and synthesis, abstraction and systematization. 2. Resolution of problems. 3. Capacity to understand advanced information in English. 4. Capacity to properly communicate both orally as in written form in Catalan, Spanish or English, in front of both expert and inexperienced audiences. 5. Capacity to understand and analyse the technological problems tied to the generation and to the modification of contents in real time. Interpersonal 6. Team work. 7. Criticism and self-criticism capabilities. 8. Exposition of results obtained. Systemic 9. Capacity to integrate knowledges and methodologies in the practical. 10. Capacity to work autonomously in the resolution of problems. 11. Concern for quality

Understand the differences and nuances of the term “real-time” in the different fields in which it applies (e.g. Computation, operating systems, media generation and processing, interaction). Understand and use correctly concepts such as control cycle, latency, granularity, jitter, etc, in real time applications. Know the MIDI and OSC protocols. Know the history, its basic principles and the possibilities of the synthesis and processing of sound. Know some of the basic technique for sound synthesis and processing, how they have evolved over time, their possibilities, and how to program it. Understand the basic techniques of spectral processing, and their possibilities Learn a programming language such as Pure Date, for the real time generation and processing of audio.

 

Assessment

For successfully passing the course, it is necessary to surpass the minimum level required for each of the competences developed in it. Given that the majority of these competences are worked and evaluated continuously in the classroom), assistance to class is considered indispensable. It is also essential that the student shows an active attitude for taking advantage of the course. This means that the student has to be critical with the aspects treated, doing his or her own reflections and keeping at all times interest and curiosity for the material presented.

The course is passed by having a minimum of 5 points in the final note. This final note will be calculated according to the activities and percentages shown in the following table (left). Given that the course is very much centered around a project, the students that have assisted and taken part actively in the classes, and that have correctly accomplished all other tasks, will not have to take the final exam. In that case, the weight for each of these activities is indicated on the second table (right).

Contribution of each of the activities in the final note (with Exam)
Compulsory labs                40% Project exposition              10% Final Project                      20% Final Examen                    30%

 

Contribution of each of the activities in the final note (without Exam)
Compulsory labs                60% Project exposition              10% Final Project                      30%

The following table synthesizes all this information

Activity	Labs and exercises	Project presentation (WIP)	Final Project	Exam
Contribution (the first value corresponds with exam; the second without)	40% [60%]	10% [10%]	20% [30%]	30% [0]
Evaluation Criteria	Per aprovar aquestes pràctiques cal lliurar-les acabades d’acord a les consignes que es donaran a les sessions pràctiques i de seminari	Per aprovar la presentació és necessari obtenir una nota superior o igual a 5. Es valorarà els continguts i la forma de la presentació oral, així com el document que la acompanyi	Per aprovar el treball final és necessari obtenir una nota superior o igual a 5	Per aprovar l'examen és necessari obtenir una nota superior o igual a 5
Minimum Requirements for passing the course	Assistència i participació activa en les sessions. Lliurament de les pràctiques acabades. En cas contrari, l’estudiant haurà de realitzar un examen final	És condició indispensable aprovar aquest treball (>=5) per poder fer mitja. En cas contrari, l’estudiant haurà de realitzar un examen final	És condició indispensable aprovar el treball (>=5) per poder fer mitja.	En cas de realitzar l’examen, és condició indispensable obtenir una nota superior o igual a 4 en l’examen per poder fer mitja
Minimum Requirements for recovery	Not recoverable	Not recoverable		Per presentar-se a recuperació es necessari haver presentat totes les pràctiques del curs. En cas de no haver lliurat alguna de les activitats, l’alumne no podrà utilitzar el període de recuperació
Recovery activity			Correcció del treball presentat o elaboració d’un nou treball	Examen teòric
Minimum Requirements for passing the course(recovery)			Aprovar el treball amb una qualificació mínima de 5 sobre 10	Aprovar l’examen teòric amb una qualificació mínima de 5 sobre 10

 

Contents

The course is organized in a theoretical block and a practical block. Following we detail the contents of each of these two blocks.

Theoretical block

Introduction to real-time, real-time architectures and basic technical aspects: What it is and what it is not the real time. The control cycle in detail. Latency. Input and output rate. Jitter. Synchronous vs. asynchronous models. Single vs. multithreaded systems. “Polling” vs. Interruptions. Buffering.

Introduction to the synthesis of audio: Oscillators and waveforms. Modulation and low frequency oscillators(LFO). Envelopes. Filters. Waveshaping.

Musical protocols and its applications: Control vs. Audio. MIDI. Open Sound Control (OSC).

The Sampler and wave files: History. Memory vs. hard disk audio. Pitch shifting and looping.

Time-based effects: Delay lines. Feedback and comb filtering. Flanger and chorus. Karplus-Strong synthesis. Live sampling.

Spectral Processing: Introduction. Basic applications: EQ, convolution, pitch-shifting, time-stretching. Analyses, classification and other advanced applications.

Practical block

1. Real-time musical programming with Pure Data.
2. Control flow vs. audio flow. Scheduling, synchrony and management of threads. GUI. Encapsulation.
3. Data Entry. MIDI and OSC communication. Basic control flow exercises.
4. Introduction to audio synthesis. Modulation and subtractive synthesis.
5. Audio processing. “Live electronics”. Time-based effects. Delays.
6. Spectral processed and advanced analysis of audio. Pitch and envelope followers. Analysis and processed spectral.

Software used
we will be using Pure Data, an open-source and multi-platform programming language, which can be used under Linux, Windows, OSX or Android.

 

Methodology

This course combines combines lectures given by the professors, with individual and group work by the students in smaller group sessions. In particular, the work has been organized in the following manner:

Theoretical lectures: given by the professors in theory classes, and addressing each of the subjects covered in the course. Students are expected to participate making questions and comments.

Seminar and practical sessions on the computer: these are sessions done in reduced groups (2 groups of practices/seminars), in which the students work individually or in groups, depending of the activities posed by the teachers. The activities posed in the seminars are of diverse character and allow to practice, review and develop actively the questions worked on the theoretical classes. To prepare these seminars, students will have to develop some required exercises, which will be implemented in groups of two or three students.

Practical final project: After the compulsory practices, the students will have to develop a small practical project, employing the knowledges and the tools studied during the course. This activity will be developed in groups of two or three students. These projects will submitted once finished the classes, in the period of examinations, but previously, in the last week of class, students will have to present orally the project to the class, as a Work in Progress (WIP). If this presentation and the subject of the project are considered satisfactory, and if the students have submitted also satisfactorily all the exercises, they will not have to take the final examination.

Final examination: A written evaluation that may combine test type questions, theoretical subjects to develop, as well as small problems of programming . Given that this course has an important practical weight, the students that have satisfied all the other tasks will be able to remain freed of examination.

Dedication of the students
The course has 4 credits ECTS that correspond to 100 hours of work of the student, of which 36 are in class. These 36 hours are divided in lectures (18 hours), and practical sessions and seminars (18 hours). The following tables detail this aspect, together with the dear dedication of the students out of class, for each one of the blocks to be evaluated.

 

	Hours in class			Hours out of class
Content Blocks	Whole group	half group
Preparation Practice 1		4		8
Preparation Practice 2		4		8
Preparation Practice 3		4		8
Lectures	14
Final project	4	6		32
Exam				8
Total:	18	18		64	Total: 100

 

Resources

Bibliography and on-line material

• Puckette, M. Theory and Techniques of Electronic Music, available on-­‐line at http://crca.ucsd.edu/~msp/techniques/latest/book.pdf
• Jordà, S. Guía de programación en PD, available on-line at http://www.tecn.upf.es/%7Esjorda/PD/IntroduccionPD3.pdf
• Pure Data Floss Manual, available on-­‐line at http://en.flossmanuals.net/puredata