Kanav Kahol Website

Course Description

Design, use and applications of multimedia systems. An introduction to acquisition, compression, storage, retrieval, and presentation of data from different media such as images, text, video and audio.
Course Content

1. Introduction to various media types: still images, video images, audio and speech, text and haptics.
2. Multimedia data acquisition, preprocessing.
3. Modeling and Storage issues
4. Introduction to Programming Libraries: DirectShow API and Java Media Framework
5. Compression techniques for audio, video and images
6. Analysis and indexing of multimedia objects
7. Multimedia object retrieval
8. Presentation and display of multimedia information
9. Communications issues
10. Applications

Grading and Activities

* Individual Projects (3)
o Project0: Programming with Direct X 5%
o Project1: Image Enhancements 10%
o Project2: Face Recognition 15%
* Homework(3) 30%
o Homework 1: Multimedia Basics, Filters
o Homework 2: Image Creation and editing using photoshop and compression.
o Homework 3: Design of Multimedia Application: Example Biometrics application.
* Midterm (3/11/2004) : 20%
* Final (5/6/2004): 20%

Topics

* Introduction
o What is multimedia?
o Introduction to various media Types
* Multimedia data Basics
o Preliminaries of signals and functions
o Introduction to Color Models
o Images and videos
o Audio
o Data Acquisition and PreProcessing
o Normal Acquisition
o Text
o Video
o Image
o Audio
o Digitization: Sampling, Quantization and Coding
o PCM
o CCD based Device and Digital Cameras
o Digitization
o Filters and their properties
o Image Operations
* Multimedia Data Libraries
o DirectShowAPI
o Java Media Framework
* Compression
o Compression Techniques
o Run length encoding
o Quadtrees
o Fractals
o Entropy coding
o Transform coding
o JPEG
o DCT & Quantization in JPEG
o Quantization Tables
o Source Coding and Entropy Coding
o MPEG
o Frame types
o Motion Compensation
o Scalability in coding
o audio
o ADPCM
o MPEG Audio Compression
o MP3
* Analysis of Multimedia Object
o Image Content Analysis
o Cognitive and Perceptual methodologies
o Content Based Image/Video Retrieval Evaluation
o Feature Extraction
o Color
o Histograms
o Techniques for reducing complexity
o Spatial color distribution
o Texture
o Object Segmentation
o Sketch
o Retrieval by Sketch
o Spatial Relationships
o 2-D Strings
o Domain Dependant Feature Analysis
o Video Content Analysis
o Video Feature Extraction
o Scene cut detection
o Camera Operations
o Motion Analysis in Digital Video
o Speech and audio Analysis:
o Content Based Audio Retrieval
o Sound Features
o Applications in Speech Recognition and synthesis
o Co-occurrence in multimedia: exploitation of events in different medias
o Semantic Web Mining.
* Storage
o Storage of MM Object
o Large quantities of data
o Hierarchical Storage System
o Data Storage devices
o CD-ROM
o DVD
o Multimedia Servers
o Continuous Media Servers
o Multimedia Data Warehousing
* Querying
o Language for MM systems
o Multimedia Documents
o Multimedia Synchronization
o Production-level Synchronization
o Presentation-level Synchronization
o Interval-based Specification
o Inter-stream Synchronization
o Models for play-out specification
o Query Formulation
o Query optimization
o Multimedia Object Retrieval
o Media Dependent Operators
* Communication
o Multimedia over network
o The OSI Model
o The OSI Layers
o Communication Issues
o Delay Jitters
o Windows Media Player
o Real Player
* Related Technologies
o Human Computer Interaction
o Ubiquitous Computing
o Wearable Computing
o Biological Computing

Projects

The objective of the projecst is to give the students hands on experience in programming with multimedia libraries and solving real world multimedia problems

Project 0: Reading a JPEG File in DirectShow

Project Objectives: To familiarize the student with the directshow API

Project 1: Design of Image Processing Averaging and Differentiation filters.

Project Objectives: To let students have hands on experience with image processing filters and their application.

Project 2: Face Recognition using eigen faces

Project Objectives: To let students have hand on experience in pattern recognition fundamentals and solving real world multimedia projects

Homeworks

Homework 1: Sampling Quantization, Design and analaysis of image filters.

Homework 2: Image Creation and Editing using photoshop.

Homework 3: Design of a complete biometric system containing face, gait and fingerprint recognition.