Kanav Kahol Website

Office Hours: Tuesday 2-4pm, Thursday by appointment

Course Material: available from http://myasucourses.asu.edu Previous offering’s course material available from www.public.asu.edu/~kkahol in the teaching section. Evaluations are also available from that session. There is no book prescribed for this class. However material will be made available to you in class or in library reserve.

Course Description:
Design, use and applications of multimedia systems. An introduction to acquisition, compression, storage, retrieval, and presentation of data from different media such as images, text, video and audio.

Course Content:

1. Introduction to various media types: still images, video images, audio and speech, text.
2. Multimedia data acquisition, preprocessing.
3. Modeling and Storage issues
4. Compression techniques for audio, video and images
5. Analysis and indexing of multimedia objects
6. Multimedia object retrieval
7. Presentation and display of multimedia information
8. Communications issues
9. Applications

Grading and Activities:

? Individual Projects (2)
o Project1 10%
o Project2 15%
? Homework(3) 30%
? Midterm (3/11/2005) : 20%
? Final (5/9/2005): 25%

Note on Cheating
While discussions in between students is encouraged, cheating will not be tolerated and be severely dealt with. On submission of a homework or project you assume complete responsibility that it is your own work and have not represented someone else’s work as yours. You also assume responsibility that no one has knowingly or unknowingly represented your work as theirs. For more information please visit these websites.

• Studentlife's Student Academic Integrity Policy:
http://www.asu.edu/studentlife/judicial/integrity.html
• ASU's policy on Academic Dishonesty in the Student Code of Conduct:
http://www.asu.edu/aad/manuals/sta/sta104-01.html
• ECET’s Academic Integrity Information Page:
http://ecet.east.asu.edu/~ecet/AcademicIntegrity/cheating.html
One ramification of this policy is that every student must assure that neither an electronic nor hard copy of their work gets into the hands of another student. You must know how to use access control to protect your files and you may not share a computing system that does not have access control with another student in this class, without taking special steps to ensure privacy of your files. If someone else in the class steals your homework (with or without your knowledge,) then you may both get failing marks for the course.

Topics

1. Introduction
a. What is multimedia?
b. Introduction to various media Types
2. Multimedia data Basics
a. Preliminaries of signals and functions
b. Introduction to Color Models
c. Images and videos
d. Audio
3. Data Acquisition and PreProcessing
a. Normal Acquisition
i. Text
ii. Video
iii. Image
iv. Audio
b. Digitization: Sampling, Quantization and Coding
i. PCM
ii. CCD based Device and Digital Cameras
iii. Digitization
iv. Filters and their properties
v. Image Operations
4. Multimedia Data Libraries
a. DirectShowAPI
b. Intel OpenCV
5. Compression
a. Compression Techniques
b. Run length encoding
c. Quadtrees
d. Fractals
e. Entropy coding
f. Transform coding
g. JPEG
i. DCT & Quantization in JPEG
ii. Quantization Tables
iii. Source Coding and Entropy Coding
h. MPEG
i. Frame types
ii. Motion Compensation
iii. Scalability in coding
i. audio
i. ADPCM
ii. MPEG Audio Compression
iii. MP3

6. Analysis of Multimedia Object
a. Image Content Analysis
i. Cognitive and Perceptual methodologies
ii. Content Based Image/Video Retrieval Evaluation
b. Feature Extraction
i. Color
ii. Histograms
iii. Techniques for reducing complexity
iv. Spatial color distribution
c. Texture
d. Object Segmentation
e. Sketch
i. Retrieval by Sketch
f. Spatial Relationships
i. 2-D Strings
g. Domain Dependant Feature Analysis
h. Video Content Analysis
i. Video Feature Extraction
j. Camera Operations
k. Motion Analysis in Digital Video
l. Speech and audio Analysis:
i. Content Based Audio Retrieval
ii. Sound Features
iii. Applications in Speech Recognition and synthesis
m. Co-occurrence in multimedia: exploitation of events in different medias
i. Semantic Web Mining.
7. Storage
a. Storage of MM Object
b. Large quantities of data
c. Hierarchical Storage System
d. Data Storage devices
i. CD-ROM
ii. DVD
iii. Multimedia Servers
iv. Continuous Media Servers
e. Multimedia Data Warehousing
8. Querying
a. Language for MM systems
b. Multimedia Documents
c. Multimedia Synchronization
d. Production-level Synchronization
e. Presentation-level Synchronization
f. Interval-based Specification
g. Inter-stream Synchronization
h. Models for play-out specification
i. Query Formulation
i. Query optimization
j. Multimedia Object Retrieval
k. Media Dependent Operators
l. An Example Schema
9. Communication
a. Multimedia over network
b. The OSI Model
c. The OSI Layers
d. Communication Issues
e. Delay Jitters
f. Windows Media Player
g. Real Player
10. Related Technologies
a. Human Computer Interaction
b. Ubiquitous Computing
c. Biological Computing
Projects
All projects will have extra-credit opportunity and will give students experience in basic image, audio and video operation. They will be due by 11:59 pm on the due date on the digital drop box.
Project 1
Application of various types of image filters to face images. Edge Detected images, and other features

Project 2
Face Recognition based on original images of faces and filtered images and comparative performance of each.

Homeworks

Homeworks will be due in class unless stated otherwise.

Homework 1
Fundamentals of signal processing, filtering, sampling and quantization

Homework 2
Image Creation using adobe Photoshop

Homework 3
Multimedia Information Systems Design: Design of Human Computer Interfaces.

Tentative Schedule
Date Day Topic Projects and Homeworks
1/19/05 w Course Introduction
1/21/05 f Multimedia Introduction, Image and video basics
1/24/05 m Multimedia Introduction, Image and video basics Preliminaries
1/26/05 w Preliminaries
1/28/05 f Acquisition of Multimedia data: Sampling and Quantization (math) Homework 1 assigned
1/31/05 m Acquisition of Multimedia data: Audio
2/2/05 w Acquisition of Multimedia data: Images
2/4/05 f Acquisition of Multimedia Data: Images and Video
2/7/05 m Acquisition of Multimedia Data: Text
2/9/05 w Acquisition of Multimedia Data: Text Introduction to Haptic Devices
2/11/05 f Image Enhancement
2/14/05 m Image Enhancement-II
2/16/05 w Image Filters-spatial domain techniques
2/18/05 f Image Filters-spatial domain techniques-II Project 1 assigned
2/21/05 m Image Filters- frequency domain techniques
2/23/05 w Image Filters- frequency domain techniques-II
2/25/05 f Image Filters- frequency domain techniques-III
2/28/05 m Image Segmentation-I Homework 1 Due
3/2/05 w Image Segmentation-II
3/4/05 f Image Segmentation-III
3/7/05 m Image Description Project 2 assigned
3/9/05 w Image Description-II Homework 2 Assigned
3/11/05 f MidTerm
3/14/05 m Spring Break
3/16/05 w Spring Break
3/18/05 f Spring Break
3/21/05 m Audio Filters and Analysis
3/23/05 w Audio Filters and Analysis
3/25/05 f Video Feature and Analysis
3/28/05 m Video Feature and Analysis Project 1 Due
3/30/05 w Text Analysis
4/1/05 f co-occurrence analysis and multimedia Homework 2 due
4/4/05 m Multimedia Data Presentation: challenges and paradigms Homework 3 assigned
4/6/05 w Compression Basics
4/8/05 f Compression Basics
4/11/05 m Compression Basics
4/13/05 w Compression JPEG, JPEG2000
4/15/05 f Compression JPEG, JPEG2000
4/18/05 m Compression MPEG4, MPEG7
4/20/05 w H.261 H.263 Discussion of Industry Standards
4/22/05 f Compression mp3, ADPCM, MPEG Audio Project 2 due
4/25/05 m Compression mp3, ADPCM, MPEG Audio
4/27/05 w Communication-I
4/29/05 f Communication-II Homework 3 Due
5/2/05 m Related Technologies
5/4/05 w reading day
5/9/05 m Final Exam 12:20pm-2:10pm