Frequently Asked Questions (FAQ) about the "LIVE555 Streaming Media" libraries

Help support improvements and extensions to the "LIVE555 Streaming Media" software: LIVE555 Funded Projects.

General questions

1. What is the typical control flow within an application that uses these libraries, and what is the role of the various "Source" and "Sink" classes in the "liveMedia" library?
2. How can I use this code within a specialized environment (such as an embedded system, or a GUI toolkit)?
3. Is this code 'thread safe'? I.e., can it be accessed by more than one thread at the same time?
4. How many concurrent connections/streams can a RTSP server (built using our code) support?
5. When I use a receiver application (that uses the "LIVE555 Streaming Media" code) to receive an incoming RTP/UDP (or raw-UDP) stream, I see significant network packet loss. Can anything be done to improve this?
6. Is there any more documentation for these libraries?
7. I would like to see new feature X added to the code. How soon can you do this for me?

Questions about the libraries' source code

1. Where is the latest version of the library code? What version of the library do I currently have?
2. What is the copyright on the source code, and how is it licensed? What are my obligations under this license?
3. What is the best way to modify or extend the functionality of the code?
4. I want to subclass one of the supplied C++ classes, but I can't because some definitions that I need are "private:" rather than "protected:". What can I do about this?
5. Do you make available or support old versions of the library?
6. Why do you not make the code available under a 'source code repository'?

Questions about the test programs (and using their code as a model for your own applications)

1. I have successfully used the "testRTSPClient" demo application to receive a RTSP/RTP stream. Using this application code as a model, how can I decode the received video (and/or audio) data?
2. When I try to receive a stream using the "openRTSP" command-line client, the RTSP protocol exchange appears to work OK, but the resulting data file(s) are empty. What's wrong?
3. The "test*Streamer" test programs read from a file. Can I modify them so that they take input from a H.264, H.265 or MPEG encoder instead, so I can stream live (rather than prerecorded) video and/or audio?
4. But what about the "testOnDemandRTSPServer" test program (for streaming via unicast)? How can I modify it so that it takes input from a live source instead of from a file?
5. Can the RTSP server implementation (e.g., as demonstrated by the "testOnDemandRTSPServer" test program) stream to set-top boxes (STBs)?
6. Does the RTSP implementation (client and/or server) support 'trick mode' operations (i.e., seek, fast-forward, reverse play)?
7. The "test*Streamer" and "test*Receiver" test programs use multicast. Can I modify them to use unicast instead?
8. For many of the "test*Streamer" test programs, the built-in RTSP server is optional (and disabled by default). For "testAMRudioStreamer", "testMPEG4VideoStreamer", "testH264VideoStreamer", "testH265VideoStreamer", and "testWAVAudioStreamer", however, the built-in RTSP server is mandatory. Why?
9. Where can I find an example of a MPEG-4 Elementary Stream video file that I can use (as "test.m4e") in the "testMPEG4VideoStreamer" or "testOnDemandRTSPServer" demo applications (or "live555MediaServer")?
10. Where can I find an example of a H.264 Elementary Stream video file that I can use (as "test.264") in the "testH264VideoStreamer" or "testOnDemandRTSPServer" demo applications (or "live555MediaServer")?
11. Where can I find an example of a H.265 Elementary Stream video file that I can use (as "test.265") in the "testH265VideoStreamer" or "testOnDemandRTSPServer" demo applications (or "live555MediaServer")?
12. Where can I find an example of a AAC Audio (ADTS format) file that I can use (as "test.aac") in the "testOnDemandRTSPServer" demo application (or "live555MediaServer")?
13. How can I stream JPEG video via RTP? There is no demo application for this.
14. When I ran "testMPEG1or2VideoStreamer", I saw several error messages like "saw unexpected code 0x000001e0". What's wrong?
15. When I stream a MP3 file (using "testMP3Streamer" or "testOnDemandRTSPServer"), I find that QuickTime Player will not play the stream. What's wrong?
16. The calls to "doEventLoop()" in the test programs do not return. How can I arrange for "doEventLoop" to return - e.g., if the user clicks on a "stop" button in a GUI?
17. The event loop implementation provided in the source code (i.e., "BasicTaskScheduler") can handle file/socket I/O events, or delayed (or periodic) tasks. How can I have the event loop handle other kinds of event (perhaps signaled from a separate thread, e.g., running a user interface)?
18. I tried using one of the test programs to stream my file, but it didn't work. Why?
19. The test programs worked OK for me, but then I modified one of them, and it no longer works. What's wrong?

Questions about RTP, RTSP, and/or SIP

1. I tried to play a "rtsp://" URL (using testRTSPClient, openRTSP, VLC, or MPlayer), or a "sip:" URL (using playSIP), but I got an error message "RTP payload format unknown or not supported". Why?
2. Why do most RTP sessions use separate streams for audio and video? How can a receiving client synchronize these streams?
3. But I notice that there's an abrupt change in a stream's presentation times after the first RTCP "SR" packet has been received. Is this a bug?
4. I am developing a RTP receiver (client). I need access to incoming packets' RTP timestamps.
5. I have a general question about RTP/RTCP, RTSP, or SIP - not specifically related to the LIVE555 Streaming Media software. Where can I get more information?

Questions about the "live-devel" mailing list

1. How do I ask questions about the "LIVE555 Streaming Media" software (including the "LIVE555 Media Server", the "LIVE555 Proxy Server", and the "LIVE555 HLS Proxy")?
2. Why do I need to subscribe to the mailing list before I can send to it?
3. When I posted a message to the mailing list, I received a response saying that my message was being moderated. Why?
4. Why do you discriminate against people who use unprofessional email addresses ("@gmail.com" etc.)?
5. Why did nobody answer my question?
6. I posted a question to the mailing list, but nobody answered it. Can I post it again?

What is the typical control flow within an application that uses these libraries, and what is the role of the various "Source" and "Sink" classes in the "liveMedia" library?

    while (1) {
        find a task that needs to be done (by looking on the delay queue,
                and the list of network read handlers);
        perform this task;
    }

    someSinkObject->startPlaying();

Data passes through a chain of 'source's and 'sink's - e.g.,

    'source1' -> 'source2' (a filter) -> 'source3' (a filter) -> 'sink'

Whenever a object (a 'sink' or one of the intermediate filter 'source's) wants to get more data, it calls "FramedSource::getNextFrame()" on the object that's to its immediate left. This is implemented by the pure virtual function "FramedSource::doGetNextFrame()", that is implemented by each 'source' object.

Each 'source' object's implementation of "doGetNextFrame()" works by arranging for an 'after getting' function to be called (from an event handler) when new data becomes available for the caller.

Note that the flow of data from 'sources' to 'sinks' happens within each application, and doesn't correspond to the sending or receiving of network packets. For example, a server application (such as "testMP3Streamer" or "testOnDemandRTSPServer") that transmits RTP packets will do so using one or more "RTPSink" objects. Each "RTPSink" object will receive data from a chain of other, 'source' or 'filter' objects (e.g., to read data from a file), and, as a side effect, transmit RTP packets. Similarly, a client application (such as "testMP3Receiver" or "openRTSP") that receives RTP packets will do so using one or more "RTPSource" objects. Each "RTPSource" object will receive RTP packets from the network, and pass the network data (without network headers) through a chain of (zero or more) 'filter' objects, ending with a 'sink' object that processes the data - e.g., by writing it to a file ("FileSink"), or by decoding and rendering the audio or video data.

How can I use this code within a specialized environment (such as an embedded system, or a GUI toolkit)?

In particular, to use the code within a GUI toolkit, your "TaskScheduler" subclass's implementation of "doEventLoop()" should be integrated with the GUI toolkit's own event handling mechanism.

Is this code 'thread safe'? I.e., can it be accessed by more than one thread at the same time?

Therefore, although it's true to say that the code is not 'thread safe', it's also somewhat misleading. It's like saying that a high-speed rail carriage is not 'airworthy'.

Longer answer: More than one thread can still use this code, if only one thread runs the library code proper, and the other thread(s) communicate with the library only via sockets, by setting global 'flag' variables (such as event loop 'watch variables'), or by calling 'event triggers'. (Note that "triggerEvent()" is the only LIVE555 function that may be called from an external (i.e., non-LIVE555) thread.)

Another possible way to access the code from multiple threads is to have each thread use its own "UsageEnvironment" and "TaskScheduler" objects, and thus its own event loop. The objects created by each thread (i.e., using its own "UsageEnvironment") must not interact (except via global variables). Such a configuration is not recommended, however; instead, it is safer to structure such an application as multiple processes, not multiple threads.

In any case, when using the "LIVE555 Streaming Media" code, you should be familiar with event-driven programming, and understand that an event-driven application can perform at least as well as one that uses threads (unless you're actually running on a multiprocessor, in which case it's usually simpler to have your application consist of multiple processes (not just threads) - one running on each processor). Note, in particular, that you do not need multiple threads in order to transmit (or receive) multiple streams concurrently.

How many concurrent connections/streams can a RTSP server (built using our code) support?

In Windows, there is also a limit set by the "FD_SETSIZE" constant, which has a default value of 64, which produces a maximum of 32 concurrent clients. You might also wish to increase this value - e.g., by redefining FS_SETSIZE at compile time.

When I use a receiver application (that uses the "LIVE555 Streaming Media" code) to receive an incoming RTP/UDP (or raw-UDP) stream, I see significant network packet loss. Can anything be done to improve this?

However, packet loss can also be caused by insufficiently large socket reception buffers in the receiver's operating system. By default, the "LIVE555 Streaming Media" code asks the operating system to allocate at least 50 kBytes of buffer memory for each incoming datagram socket. (Note the call to increaseReceiveBufferTo() in "liveMedia/MultiFramedRTPSource.cpp".) However, you can also ask to increase this buffer size, by calling increaseReceiveBufferTo() again, within your own application code. (Note that increaseReceiveBufferTo() returns the actual resulting socket buffer size, reported by the OS, so you can check the return value to verify the resulting buffer size.)

• Note, for example, the code for the VLC media player, which sets the OS's internal receive buffer to 2,000,000 bytes for the video stream, and 100,000 bytes for the audio stream - see "modules/demux/live555.cpp".
• Note: If you are using the Linux OS, you might first need to set the OS's maximum buffer size, before you call increaseReceiveBufferTo(). You do this by running (as root):

      sysctl net.core.rmem_max = the_maximum_buffer_size_that_you_need
  

It's important to understand that because a LIVE555 Streaming Media application runs as a single thread (never writing to, or reading from, sockets concurrently), if packet loss occurs, then it must be happening either (i) on the network, or (ii) in the operating system of the sender or receiver. There's nothing in our code that can be 'losing' packets.

Is there any more documentation for these libraries?

I would like to see new feature X added to the code. How soon can you do this for me?

Also, some upcoming features have been given special priority - as LIVE555 Funded Projects. Please contribute to these projects if you wish to see them become part of the released code.

Where is the latest version of the library code? What version of the library do I currently have?

Note: You should avoid using software named "live555" that you might find on other web sites (including 'software repository' sites such as GitHub). We do not endorse these versions (note that we do not provide our code from a source code repository); they may contain unknown modifications, and/or bugs (including security vulnerabilites) that have been fixed in the latest version.

To see which version of the code you currently have, look at the file "liveMedia/include/liveMedia_version.hh".

What is the copyright on the source code, and how is it licensed? What are my obligations under this license?

This FAQ is not a legal document. If you have any questions about your compliance with the LGPL and its conditions, please consult your copyright attorney. However, your obligations under the LGPL include (but are not necessarily limited to) the following:

• If you modify (i.e., make any changes to) the supplied LIVE555 library code - either the ".hh" or the ".cpp" files - and distribute a product (e.g., a network camera, media server, media player, or proxy server) that's built from this modified code, then you must also distribute your modifications (e.g., by posting your modifications on your product's web site). (As noted below, this does not apply if you have merely subclassed the LIVE555 library code, without modifying it.)

  Note that merely sending your code modifications to the "live-devel" mailing list (even in the form of a 'patch' file) is not sufficient - unless your patch happens to get included in a future release of the LIVE555 code, and you then upgrade to this new version of the code (without modifying it again). But there's no guarantee that we will accept your patch (and we are certainly under no obligation to do so). We are unlikely to accept very large patches, nor patches that add functionality that's not likely to be of general interest. (Of course, patches that fix genuine bugs are always welcome.)

• If you distribute a product (whether software or hardware; whether free or for pay) that uses the LIVE555 library code, then you must - when requested by either a customer, or Live Networks, Inc. - upgrade it as soon as possible to use the latest version of the LIVE555 library code (or else provide a way for your customers to perform this upgrade themselves). Similarly, you must allow any of your customers to - at their discretion - replace the LIVE555 libraries used by your product with their own version (that uses the same API). (Note that in particular, because Apple restricts which apps are made available on their 'App Store', you cannot use the LIVE555 library code in an app that's distributed through Apple's 'App Store'.)

  Note that if you have a hardware product that is released 'as is', without any mechanism for upgrading its firmware, then this does not absolve you of this responsibility. If you have no mechanism for upgrading your product's firmware after it is released, then your product cannot use this software.)

What is the best way to modify or extend the functionality of the code?

Note also that subclassing the code considerably simplifies your obligations under the LGPL. If you modify the supplied code, and then release a product based on these modifications, then - as noted above - you are required to also make your modified source code available. If, instead, you subclass the supplied code (without modifying it), you are not required to release your subclass code (nor the rest of your application code). Your application can be 'closed source'.

I want to subclass one of the supplied C++ classes, but I can't because some definitions that I need are "private:" rather than "protected:". What can I do about this?

Do you make available or support old versions of the library?

It's important to understand that this software - unlike some other open source projects - does not have separate 'stable' and 'experimental' releases. Instead, there's just one release, and it can be considered 'stable'.

Why do you not make the code available under a 'source code repository'?

I have successfully used the "testRTSPClient" demo application to receive a RTSP/RTP stream. Using this application code as a model, how can I decode the received video (and/or audio) data?

If you want to decode (or otherwise process) these frames, you would replace "DummySink" with your own "MediaSink" subclass. Its "afterGettingFrame()" function would pass the data (at "fReceiveBuffer", of length "frameSize") to a decoder. (A decoder would also use the "presentationTime" timestamp to properly time the rendering of each frame, and to synchronize audio and video.)

If you are receiving H.264 video data, there is one more thing that you have to do before you start feeding frames to your decoder. H.264 streams have out-of-band configuration information ("SPS" and "PPS" NAL units) that you may need to feed to the decoder to initialize it. To get this information, call "MediaSubsession::fmtp_spropparametersets()" (on the video 'subsession' object). This will give you a (ASCII) character string. You can then pass this to "parseSPropParameterSets()" (defined in the file "include/H264VideoRTPSource.hh"), to generate binary NAL units for your decoder.

(If you are receiving H.265 video, then you do the same thing, except that you have three separate configuration strings, that you get by calling "MediaSubsession::fmtp_spropvps()", "MediaSubsession::fmtp_spropsps()", and "MediaSubsession::fmtp_sproppps()". For each of these three strings, in turn, pass them to "parseSPropParameterSets()", then feed the resulting binary NAL unit to your decoder.)

When I try to receive a stream using the "openRTSP" command-line client, the RTSP protocol exchange appears to work OK, but the resulting data file(s) are empty. What's wrong?

If, instead, you're using the "testRTSPClient" demo application, note the line

    #define REQUEST_STREAMING_OVER_TCP False

The "test*Streamer" test programs read from a file. Can I modify them so that they take input from a H.264, H.265, or MPEG encoder instead, so I can stream live (rather than prerecorded) video and/or audio?

Alternatively, if your encoder presents you with a sequence of frames (or 'NAL units'), rather than a sequence of bytes, then a more efficient solution would be to write your own "FramedSource" subclass that encapsulates your encoder, and delivers audio or video frames directly to the appropriate "*RTPSink" object. This avoids the need for an intermediate 'framer' filter that parses the input byte stream. (If, however, you are streaming H.264, H.265, or MPEG-4 (or MPEG-2 video with "B" frames), then you should insert the appropriate "*DiscreteFramer" filter between your source object and your "*RTPSink" object.)

For a model of how to do that, see "liveMedia/DeviceSource.cpp" (and "liveMedia/include/DeviceSource.hh"). You will need to fill in parts of this code to do the actual reading from your encoder.

But what about the "testOnDemandRTSPServer" test program (for streaming via unicast)? How can I modify it so that it takes input from a live source instead of from a file?

Then, as above, if your input device is accessible by a file name (including "stdin" for standard input), then simply replace the appropriate "test.*" file name with the file name of your input device.

If, however, you have written your own "FramedSource" subclass (e.g., based on "DeviceSource", as noted above) to encapsulate your input source, then the solution is a little more complicated. In this case, you will also need to define and implement your own new subclass of "OnDemandServerMediaSubsession" that gets its input from your live source, rather than from a file. In particular, you will need to provide your own implementation of the two pure virtual functions "createNewStreamSource()" and "createNewRTPSink()". For a model of how to do this, see the existing "FileServerMediaSubsession" subclass that is used to stream your desired type of data from an input file. (For example, if you are streaming H.264 video, you would use "H264VideoFileServerMediaSubsession" as a model.) Note that:

• Your "createNewStreamSource()" implementation will create (and return) an instance of your input source object. (You should also set the "estBitrate" result parameter to be the estimated bit rate (in kbps) of the stream. This estimate is used to determine the frequency of RTCP packets; it is not essential that it be accurate.)
• Your "createNewRTPSink()" implementation will create (and return) an appropriate new "RTPSink" (subclass) object. (The code for this will usually be the same as the code for "createNewRTPSink()" in the corresponding "FileServerMediaSubsession" subclass.)

Can the RTSP server implementation (e.g., as demonstrated by the "testOnDemandRTSPServer" test program) stream to set-top boxes (STBs)?

• Many (most?) STBs do not support RTP; instead, they handle streams of raw-UDP packets. There is currently no standard way to use the RTSP protocol to request raw-UDP streams, so different STB (and server) manufacturers tend to define their own (often incompatible) RTSP extensions for this.
• STBs typically handle only MPEG Transport Streams, not Elementary Streams or other data formats. (Because MPEG Transport Streams contain their own timestamps, they can be streamed via raw-UDP, with RTP timestamps not being required.)
• Some STBs do not handle multicast streams.

Does the RTSP implementation (client and/or server) support 'trick mode' operations (i.e., seek, fast-forward, reverse play)?

Our RTSP client implementation fully supports 'trick play' operations. Note the "start", "end" and "scale" parameters to "RTSPClient::sendPlayCommand()". (Note also that our "openRTSP" demo RTSP client application has command-line options that can be used to demonstrate client 'trick play' operations.)

Our RTSP server implementation also supports 'trick play' operations, but note that parts of this are (necessarily) media type specific. I.e., there has to be some new code added for each different type of media file that we wish to stream. This functionality has already been provided for some types of media file.

To add 'trick play' support for a media type (that does not already support it), changes need to be made to the corresponding subclass of "ServerMediaSubsession":

1. To add support for seeking within a stream, you will need to implement the following virtual functions:

   • virtual float duration() const;

     Returns the file's duration, in seconds

   • virtual void seekStreamSource(FramedSource* inputSource, double& seekNPT, double streamDuration, u_int64_t& numBytes);

     (Attempts to) seek within the input source.
2. To add support for 'fast forward' and/or 'reverse play', you will also need to implement the following virtual functions:

   • virtual void testScaleFactor(float& scale);

     Inspects the input value of "scale", and, if necessary, changes it to a nearby value that we support. (E.g., if the input value of "scale" is 3.3, you might change it to 3 (an integer).) If there's no 'nearby' value that you support, just set "scale" to 1 (the default value).

   • virtual void setStreamSourceScale(FramedSource* inputSource, float scale);

     Actually sets the scale factor for a specific input source. (The value of "scale" will previously have been passed in and out of "testScaleFactor()", so we know that it is a value that we support.)

The "test*Streamer" and "test*Receiver" test programs use multicast. Can I modify them to use unicast instead?

If you still wish to change the "test*Streamer" programs to stream using unicast, then do the following:

1. In "test*Streamer.cpp", change "destinationAddressStr" to the (unicast) IP address of the intended destination.
2. In the corresponding "test*Receiver.cpp", change "sessionAddressStr" to "0.0.0.0".
3. (optional) If you also want to send RTCP packets (e.g., RTCP Receiver Reports) back to the streaming server, then you will also need to do the following - in "test*Receiver.cpp" - after you've created "rtcpGroupsock". (In this example, suppose that the streaming server has IP address "10.20.30.40" and uses port 6667 for RTCP.):

       struct in_addr serverAddress;
       serverAddress.s_addr = our_inet_addr("10.20.30.40");
       rtcpGroupsock.changeDestinationParameters(serverAddress, 6667, 255);
   

For many of the "test*Streamer" test programs, the built-in RTSP server is optional (and disabled by default). For "testAMRudioStreamer", "testMPEG4VideoStreamer", "testH264VideoStreamer", "testH265VideoStreamer", and "testWAVAudioStreamer", however, the built-in RTSP server is mandatory. Why?

Where can I find an example of a MPEG-4 Elementary Stream video file that I can use (as "test.m4e") in the "testMPEG4VideoStreamer" or "testOnDemandRTSPServer" demo applications (or "live555MediaServer")?

If you search in an online search engine for

+"rtsp://" +".mp4"

This should give you two files: "video-MP4V-ES-1" and "audio-MPEG4-GENERIC-2". (If, instead, you get a file "video-H264-1", then this is H.264 video, not MPEG-4 video. Try again with another stream.) Rename the file "video-MP4V-ES-1" as "test.m4e", and you will be able to use it in "testMPEG4VideoStreamer" and "testOnDemandRTSPServer".

We have also made some example MPEG-4 Video Elementary Stream (".m4e") files available online here.

Where can I find an example of a H.264 Elementary Stream video file that I can use (as "test.264") in the "testH264VideoStreamer" or "testOnDemandRTSPServer" demo applications (or "live555MediaServer")?

We have also made some example H.264 Video Elementary Stream (".264") files available online here.

Where can I find an example of a H.265 Elementary Stream video file that I can use (as "test.265") in the "testH265VideoStreamer" or "testOnDemandRTSPServer" demo applications (or "live555MediaServer")?

Where can I find an example of a AAC Audio (ADTS format) file that I can use (as "test.aac") in the "testOnDemandRTSPServer" demo application (or "live555MediaServer")?

How can I stream JPEG video via RTP? There is no demo application for this.

You should be aware, though, that JPEG is a very poor codec for video streaming, because (unlike MPEG, H.264 or H.265 video) there is no inter-frame compression. Every video frame is a 'key' frame, and is sent in its entirety. Also, each frame is typically large (and so takes up many network packets). If any of these network packets gets lost, then the whole frame must be discarded. JPEG video streaming is strongly discouraged, and should be considered (if at all) only for high-bitrate local-area networks with very low packet loss.

When I ran "testMPEG1or2VideoStreamer", I saw several error messages like "saw unexpected code 0x000001e0". What's wrong?

    #define SOURCE_IS_PROGRAM_STREAM 1

When I stream a MP3 file (using "testMP3Streamer" or "testOnDemandRTSPServer"), I find that QuickTime Player will not play the stream. What's wrong?

Blame Apple for this. They have known about this bug for many years, but - for some odd reason - do not consider it a high priority bug.

Instead, we recommend that you use the VLC media player.

The calls to "doEventLoop()" in the test programs do not return. How can I arrange for "doEventLoop" to return - e.g., if the user clicks on a "stop" button in a GUI?

The event loop implementation provided in the source code (i.e., "BasicTaskScheduler") can handle file/socket I/O events, or delayed (or periodic) tasks. How can I have the event loop handle other kinds of event (perhaps signaled from a separate thread, e.g., running a user interface)?

Alternatively, you could subclass "TaskScheduler" to implement your own event loop - but that is more difficult.

I tried using one of the test programs to stream my file, but it didn't work. Why?

If you're sure that your file is of the correct type, then please put the file on a publically-accessible web (or FTP) server, and post the URL (not the file itself) to the "live-devel" mailing list, and we'll take a look at it, to see if we can figure out what's wrong.

The test programs worked OK for me, but then I modified one of them, and it no longer works. What's wrong?

I tried to play a "rtsp://" URL (using testRTSPClient, openRTSP, VLC, or MPlayer), or a "sip:" URL (using playSIP), but I got an error message "RTP payload format unknown or not supported". Why?

An "RTP payload format" for a codec is a set of rules that define how the codec's media frames are packed within RTP packets. This is usually defined by an IETF RFC, or - for newer payload formats - an IETF Internet-Draft. However, a few RTP payload formats (usually those whose MIME subtype begins with "X-") are proprietary, are not defined in publically-available documents.

The "liveMedia" library supports many, but not all, RTP payload formats. If you encounter a RTP payload format that is not supported, but which is defined by a publically-available document, then we may be able to add support for it, if there is sufficient interest.

Why do most RTP sessions use separate streams for audio and video? How can a receiving client synchronize these streams?

These audio and video streams are synchronized using RTCP "Sender Report" (SR) packets - which map each stream's RTP timestamp to 'wall clock' (NTP) time. For more information, see the IETF's RTP/RTCP specification.

Receivers can then use this mapping to synchronize the incoming RTP streams. The LIVE555 Streaming Media code does this automatically: For subclasses of "RTPSource", the "presentationTime" parameter that's passed to the 'afterGettingFunc' of "getNextFrame()" (see "liveMedia/include/FramedSource.hh") will be an accurate, time-synchronized time. (For this to work, you need to have also created a "RTCPInstance" for each RTP source.)

For example, if you use "openRTSP" to receive RTSP/RTP streams, then the contents of each RTP stream (audio and video) are written into separate files. This is done using the "FileSink" class. If you look at the "FileSink::afterGettingFrame()" member function, you'll notice that there's a "presentationTime" parameter for each incoming frame. Some other receiver could use the "presentationTime" parameter to synchronize audio and video.

But I notice that there's an abrupt change in a stream's presentation times after the first RTCP "SR" packet has been received. Is this a bug?

This means is that a receiver should be prepared for the fact that the first few presentation times (until RTCP synchronization starts) will not be accurate. The code, however, can check this by calling "RTPSource:: hasBeenSynchronizedUsingRTCP()". If this returns False, then the presentation times are not accurate, and should not be used for synchronization. However, once the call to returns True, then the presentation times (from then on) will be accurate.

I am developing a RTP receiver (client). I need access to incoming packets' RTP timestamps.

I have a general question about RTP/RTCP, RTSP, or SIP - not specifically related to the LIVE555 Streaming Media software. Where can I get more information?

RTSP is standardized by the IETF's Multiparty Multimedia Session Control ("mmusic") working group. (For more information, see www.rtsp.org)

SIP is standardized by the IETF's Session Initiation Protocol ("sip") and Session Initiation Proposal Investigation ("sipping") working groups. (For more information, see Henning Schulzrinne's site.)

How do I ask questions about the "LIVE555 Streaming Media" software (including the "LIVE555 Media Server", the "LIVE555 Proxy Server", and the "LIVE555 HLS Proxy)?

        [email protected]

Why do I need to subscribe to the mailing list before I can post to it?

(Note that you must subscribe to the mailing list using the same "From:" address that intend to use to later post messages to the list.)

When I posted a message to the mailing list, I received a response saying that my message was being moderated. Why?

(If you learn that your message to the mailing list is being moderated, then please wait until it gets approved and delivered before you post another message to the list - otherwise (to prevent abuse of the mailing list) only the last such message may get approved.)

Why do you discriminate against people who use unprofessional email addresses ("@gmail.com" etc.)?

This software has always been intended for use by professional software developers: People who are affiliated with corporations using this software commercially (or schools using this software for research). As the software is open source, however, hobbyists are also free to use it, but to post to this mailing list, they are expected to demonstrate at least a minimal level of 'cluefulness' by using an email address with their own domain name - not just a generic "@gmail", "@yahoo", etc.-type email address. (Note that the underlying email service is not the problem; you can use your own domain name even with a public web-based email service like 'Gmail' or 'Yahoo Mail'.)

Why did nobody answer my question?

I posted a question to the mailing list, but nobody answered it. Can I post it again?

Once your question is posted to the mailing list (you can check this by looking at the list's archives), then rest assured that hundreds of people will get to see it. But sometimes, a question does not get answered (see above). If that happens, then sorry - but do not send the question to the list again.

Live Networks, Inc. (LIVE555.COM)