CS 지식/네트워크

[네트워크] 28. Multimedia

cdragon 2023. 2. 13. 16:17
반응형

Internet Audio/Video

  • Streaming stored audio/video refers to on-demand requests for compressed audio/video files
    • youtube, netflix
    • 저장, 압축된 영상매체를 불러오기
  • Streaming live audio/video refers to the broadcasting of radio and TV programs through the Internet
    • 뉴스, live 방송
  • Interactive audio/video refers to the use of the Internet for interactive audio/video applications
    • 카카오 보이스톡, 페이스톡, 줌
  • 각 서비스 마다 요구사항이 달라서 나누었음.

Digitizing Audio and Video

  • Digitizing audio
    • Voice: 64 Kbps = 8,000 samples x 8 bits
    • Music: 44,100 samples x 16 bits ->
    • ​ 705.6 Kbps for mono, 1,411 Mbps for stereo
  • Digitizing video
    • 25 frames/second, 1024 x 768 pixels, 24 bits/pixel
    • 2 x 25 x 1024 x 768 x 24 = 944 Mbps
  • Compression is needed to send video (audio) over the Internet
  • Audio Compression
    • Predictive encoding: GSM(13Kbps), G.729(8Kbps),
    ​ G.723.3(6.4 or 5.3 Kbps)
    • Perceptual encoding: MP3 (MPEG audio layer 3)
  • Video Compression
    • Image compression: JPEG
    • Video compression: MPEG

 

Streaming Stored AV: (1)

  • First approach: Using a Web server

  • 어딘가(서버) 저장된 비디오를 꺼내서 보는 방식
  • Get - Response
    • 가져온 content를 media player를 통해 play

 

Streaming Stored AV: (2)

  • Second approach: Using a Web server with metafile

  • meta file - content와 관련된 information
  • media player가 metafile을 보고 server에 GET 요청하여 response를 받아 직접 play

 

Streaming Stored AV: (3)

  • Third approach: Using a media server

  • 분리되어 있음

 

Streaming Stored AV: (4)

  • Fourth approach: Using a media server and RTSP

  • real-time streaming protocol
  • metafile요청
  • metafile media player로 넘김
  • RTSP는 자체적인 protocol을 가지고 있음
    • play, pause, fase-forward, rewind 등 다양한 비디오 컨트롤 기능을 가능케 함.

 

Real-time Interactive AV

live에 대해서는 더 strict한 방식을 사용해야 함.

  • Time relationship

  • Jitter is introduced in real-time data by the delay between packets

  • 각각의 packet 들이 겪게되는 delay가 다 다르다. -> jitter

 

Timestamp

  • To prevent jitter, we can timestamp the packets and separate the arrival time from the playback time
    • arrival time과 playback time을 분리
    •  

Playback Buffer

  • A playback buffer is required for real-time traffic

1초에 도착한 packet이 7초 동안 기다렸다가 playback 된다.

stop을 막을 수 있음.

 

Characteristics of Real-time AV Communication

  • A sequence number on each packet is required for real-time traffic
  • Real-time traffic needs the support of multicasting
  • Translation means changing the encoding of a payload to a lower quality to match the bandwidth of the receiving network
  • Mixing means combining several streams of traffic into one stream.
  • Support from transport layer protocol
    • TCP, with all its sophistication(복잡함), is not suitable for interactive multimedia traffic because we cannot allow retransmission of packets.
    • UDP is more suitable than TCP for interactive traffic. However, we need the services of RTP, another transport layer protocol, to make up for the deficiencies of UDP
      • RTP + UDP

 

Real-time Transport Protocol (RTP)

  • RTP is designed to handle real-time traffic on the Internet
  • RTP uses a temporary even-numbered UDP port
    • odd-number - RTCP

 

  • payload type: MPEG 등 어떤 압축 방법으로 encoding되어 있는지
  • timestamp: 이 값을 가지고 play time을 정렬

 

Real-time Transport Control Protocol (RTCP)

  • RTCP is designed to control the flow and quality of data and allow the recipient(수용자) to send feedback to the source or sources
    • source에게 feedback 전송
  • RTCP uses an odd-numbered UDP port number that follows the port number selected for RTP

Voice over IP

  • Session Initiation Protocol (SIP) by IETF and H.323 by ITU-T
  • SIP

SIP Simple Session

Tracking the Callee

H.323 Architecture and Protocols - 생략

  • SIP가 대세 VoIP

 

H.323 Example - 생략

Push/Pull Based (Adaptive) Streaming

  • Adaptive:
    • 자동차 이동 중에 동영상을 본다고 하면 계속해서 네트워크 상황이 변할 것이다.(기지국과의 거리에 따라, 음영지역의 여부에 따라)
    • 그래서 적응적으로 화질을 일부러 떨어트리거나 하는 등
  • Push based streaming (서버가 주도하여 push) - IPTV 방식
    • Session setup between server and client
    • RTSP + RTP +UDP
  • Pull based streaming (Client가 주도하여 Pull) - 대부분 사용 중인 방식
    • Contents request from client
      • 내가 play/pause
    • HTTP + TCP

 

HTTP Adaptive Streaming (HAS)

  • HTTP + TCP
  • HAS follows pull-based streaming paradigm
    • Rather than the traditional push-based streaming based on stateful protocol such as RTSP
      • state을 관리
    • Since HTTP is a stateless protocol, the client plays the central role by carrying the intelligence that drives the video adaptation
  • The advantage of HAS
    • Avoids NAT and fire-wall traversal issues - 너무 많은 traffic으로 fire-wall나 NAT에 막히는 경우 대부분 발생
    • Reusing existing infrastructure
    • SO, 추세는 RTP + UDP 에서 HTTP + TCP로 mainstream을 이루고 있다.
  • Existing versions of HAS
    • Apple : HTTP Live Streaming(HLS)
    • Microsoft : IIS Smooth Streaming
    • Adobe : HTTP Dynamic Streaming
    • DASH standardized by 3GPP(모바일을 주도하는) and MPEG
      • 국제적인 표준으로 집합된.
      • Dynamic Adaptive Streaming over HTTP
      • Netflix와 같은 곳에서 DASH 지원함.

  • 네트워크 혼잡도 상황 등을 이용해서 비디오 퀄리티 중에 네트워크 상황에 맞는 퀄리티를 pulling
    • client가 모든 intelligence를 가져 client가 요청하는 대로 플레이
    • 과거 상황이 어떻든 상관없음.

 

  • 계속해서 quality를 늘려가다가 loss나 congestion이 탐지 되면 다시 또 떨어 뜨리고
  • 2초 짜리의 segment
    • 2초마다 client 요청에 따라 quality의 movie를 request

 

반응형