[네트워크] 28. Multimedia

Internet Audio/Video

• Streaming stored audio/video refers to on-demand requests for compressed audio/video files
  • youtube, netflix
  • 저장, 압축된 영상매체를 불러오기
• Streaming live audio/video refers to the broadcasting of radio and TV programs through the Internet
  • 뉴스, live 방송
• Interactive audio/video refers to the use of the Internet for interactive audio/video applications
  • 카카오 보이스톡, 페이스톡, 줌
• 각 서비스 마다 요구사항이 달라서 나누었음.

Digitizing Audio and Video

• Digitizing audio
  • Voice: 64 Kbps = 8,000 samples x 8 bits
  • Music: 44,100 samples x 16 bits ->
  • ​ 705.6 Kbps for mono, 1,411 Mbps for stereo
• Digitizing video
  • 25 frames/second, 1024 x 768 pixels, 24 bits/pixel
  • 2 x 25 x 1024 x 768 x 24 = 944 Mbps
• Compression is needed to send video (audio) over the Internet
• Audio Compression
  • Predictive encoding: GSM(13Kbps), G.729(8Kbps),
  ​ G.723.3(6.4 or 5.3 Kbps)
  • Perceptual encoding: MP3 (MPEG audio layer 3)
• Video Compression
  • Image compression: JPEG
  • Video compression: MPEG

 

Streaming Stored AV: (1)

• First approach: Using a Web server

• 어딘가(서버) 저장된 비디오를 꺼내서 보는 방식
• Get - Response
  • 가져온 content를 media player를 통해 play

 

Streaming Stored AV: (2)

• Second approach: Using a Web server with metafile

• meta file - content와 관련된 information
• media player가 metafile을 보고 server에 GET 요청하여 response를 받아 직접 play

 

Streaming Stored AV: (3)

• Third approach: Using a media server

• 분리되어 있음

 

Streaming Stored AV: (4)

• Fourth approach: Using a media server and RTSP

• real-time streaming protocol
• metafile요청
• metafile media player로 넘김
• RTSP는 자체적인 protocol을 가지고 있음
  • play, pause, fase-forward, rewind 등 다양한 비디오 컨트롤 기능을 가능케 함.

 

Real-time Interactive AV

live에 대해서는 더 strict한 방식을 사용해야 함.

• Time relationship

• Jitter is introduced in real-time data by the delay between packets

• 각각의 packet 들이 겪게되는 delay가 다 다르다. -> jitter

 

Timestamp

• To prevent jitter, we can timestamp the packets and separate the arrival time from the playback time
  • arrival time과 playback time을 분리
  •  

Playback Buffer

• A playback buffer is required for real-time traffic

1초에 도착한 packet이 7초 동안 기다렸다가 playback 된다.

stop을 막을 수 있음.

 

Characteristics of Real-time AV Communication

• A sequence number on each packet is required for real-time traffic
• Real-time traffic needs the support of multicasting

• Translation means changing the encoding of a payload to a lower quality to match the bandwidth of the receiving network
• Mixing means combining several streams of traffic into one stream.
• Support from transport layer protocol
  • TCP, with all its sophistication(복잡함), is not suitable for interactive multimedia traffic because we cannot allow retransmission of packets.
  • UDP is more suitable than TCP for interactive traffic. However, we need the services of RTP, another transport layer protocol, to make up for the deficiencies of UDP
    • RTP + UDP

 

Real-time Transport Protocol (RTP)

• RTP is designed to handle real-time traffic on the Internet
• RTP uses a temporary even-numbered UDP port
  • odd-number - RTCP

 

• payload type: MPEG 등 어떤 압축 방법으로 encoding되어 있는지
• timestamp: 이 값을 가지고 play time을 정렬

 

Real-time Transport Control Protocol (RTCP)

• RTCP is designed to control the flow and quality of data and allow the recipient(수용자) to send feedback to the source or sources
  • source에게 feedback 전송
• RTCP uses an odd-numbered UDP port number that follows the port number selected for RTP

Voice over IP

• Session Initiation Protocol (SIP) by IETF and H.323 by ITU-T
• SIP

SIP Simple Session

Tracking the Callee

H.323 Architecture and Protocols - 생략

• SIP가 대세 VoIP

 

H.323 Example - 생략

Push/Pull Based (Adaptive) Streaming

• Adaptive:
  • 자동차 이동 중에 동영상을 본다고 하면 계속해서 네트워크 상황이 변할 것이다.(기지국과의 거리에 따라, 음영지역의 여부에 따라)
  • 그래서 적응적으로 화질을 일부러 떨어트리거나 하는 등
• Push based streaming (서버가 주도하여 push) - IPTV 방식
  • Session setup between server and client
  • RTSP + RTP +UDP
• Pull based streaming (Client가 주도하여 Pull) - 대부분 사용 중인 방식
  • Contents request from client
    • 내가 play/pause
  • HTTP + TCP

 

HTTP Adaptive Streaming (HAS)

• HTTP + TCP
• HAS follows pull-based streaming paradigm
  • Rather than the traditional push-based streaming based on stateful protocol such as RTSP
    • state을 관리
  • Since HTTP is a stateless protocol, the client plays the central role by carrying the intelligence that drives the video adaptation
• The advantage of HAS
  • Avoids NAT and fire-wall traversal issues - 너무 많은 traffic으로 fire-wall나 NAT에 막히는 경우 대부분 발생
  • Reusing existing infrastructure
  • SO, 추세는 RTP + UDP 에서 HTTP + TCP로 mainstream을 이루고 있다.
• Existing versions of HAS
  • Apple : HTTP Live Streaming(HLS)
  • Microsoft : IIS Smooth Streaming
  • Adobe : HTTP Dynamic Streaming
  • DASH standardized by 3GPP(모바일을 주도하는) and MPEG
    • 국제적인 표준으로 집합된.
    • Dynamic Adaptive Streaming over HTTP
    • Netflix와 같은 곳에서 DASH 지원함.

• 네트워크 혼잡도 상황 등을 이용해서 비디오 퀄리티 중에 네트워크 상황에 맞는 퀄리티를 pulling
  • client가 모든 intelligence를 가져 client가 요청하는 대로 플레이
  • 과거 상황이 어떻든 상관없음.

 

• 계속해서 quality를 늘려가다가 loss나 congestion이 탐지 되면 다시 또 떨어 뜨리고
• 2초 짜리의 segment
  • 2초마다 client 요청에 따라 quality의 movie를 request