SEI Insertion
OvenMediaEngine Enterprise supports SEI insertion starting from version 0.18.0.0-1.
Last updated
OvenMediaEngine Enterprise supports SEI insertion starting from version 0.18.0.0-1.
Last updated
OvenMediaEngine can insert Supplemental Enhancement Information (SEI) into live streams to deliver custom data with video content at frame-level precision.
You can insert SEI dynamically through the OvenMediaEngine's Send Event API or continuously through XML configuration.
When inserting SEI, you can add custom data, and the inserted SEI automatically includes UUID
and Timestamp
.
The inserted SEI is generated in OvenMediaEngine-specific format containing UUID, Timestamp, and custom data.
OvenPlayer can receive SEI inserted by OvenMediaEngine and provides functionality to automatically parse the SEI specification defined by OvenMediaEngine.
You can use OvenMediaEngine's SendEvent REST API to dynamically trigger events that insert SEI into streams.
Request
eventFormat
Y
Specifies the event format (use sei
format).
eventType
N
Specifies the event type.
Default: video
events
Y
Contains event data values.
event.seiType
N
Specifies SEI type.
Default: UserDataUnregistered
event.data
Y
Enter the actual data to be transmitted.
For scenarios requiring continuous SEI insertion, you can configure this behavior through XML configuration. Create an XML file defining SEI insertion events and enable EventGenerator
in Server.xml
.
Server.xml: You can enable the EventGenerator functionality for SEI insertion by adding <Application><EventGenerator>
.
<?xml version="1.0" encoding="UTF-8"?>
<Server version="8">
...
<VirtualHosts>
<VirtualHost>
<Applications>
<Application>
...
<EventGenerator>
<Enable>true</Enable>
<Path>events/send_event_info.xml</Path>
</EventGenerator>
</Application>
</Applications>
</VirtualHost>
</VirtualHosts>
</Server>
<Enable>
Y
Sets activation status to true
or false
.
Default: false
<Path>
Y
Sets the path to the XML file defining SEI insertion details. If a relative path is specified, the directory containing the Server.xml
file is used as the base.
XML Defining SEI Insertion Events: Create an XML file defining SEI insertion events at the path specified in Server.xml
. In this example, it's send_event_info.xml
.
<?xml version="1.0" encoding="UTF-8"?>
<EventInfo>
<Event>
<Enable>true</Enable>
<SourceStreamName>stream*</SourceStreamName>
<Interval>2000</Interval>
<EventFormat>sei</EventFormat>
<EventType>video</EventType>
<Values>
<SeiType>UserDataUnregistered</SeiType>
<Data>Hi! OvenMediaEngine! CurrentTime:${EpochTime}</Data>
<KeyframeOnly>true</KeyframeOnly>
</Values>
</Event>
</EventInfo>
<Enable>
Y
Sets activation status to true
or false
.
Default: false
<SourceStreamName>
Y
Specifies the target stream name.
It supports wildcards (*
) for pattern matching.
<Interval>
Y
Sets event occurrence interval in milliseconds (ms).
<EventFormat>
Y
Specifies event format (use sei
format).
<EventType>
N
Specifies the event type.
Default: video
<Values>
The payload of SEI generated by OvenMediaEngine always includes UUID and Timestamp values in the following structure:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| uuid_iso_iec_11578(128) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp (64) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Data (Payload Size - UUID(128) - Timestamp(64)) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
UUID
128
Indicates that the SEI Payload follows OvenMediaEngine's specification. Always set to 464d4c47-5241-494e-434f-4c4f554201
Timestamp
64
Epoch time in milliseconds
Data
Varies depending on custom data
Custom data
OvenPlayer can parse SEI inserted by OvenMediaEngine and pass the included UUID, Timestamp, and custom data to the application.
var player = OvenPlayer.create('player', {
sources: [
{
type: 'webrtc', // Playing WebRTC stream
file: 'wss://[YOUR_OvenMediaEngine]:3333/app/stream'
}
],
parseStream: {
enabled: true // Enable H.264 NAL parsing
}
});
function toAsciiString(byteArray) {
return String.fromCharCode.apply(null, byteArray);
}
player.on('metaData', function (metadata) {
console.log('MetaData:', metadata);
/* Output:
{
type: 'sei',
nalu: Unit8Array(33),
sei: {
type: 5,
size: 39,
payload: Unit8Array(39)
},
registered: true,
uuid: '464d4c47-5241-494e-434f-4c4f-55524201',
timecode: 1739851602778,
userdata: Unit8Array(15)
}
*/
console.log(`Convert user data to string: ${toAsciiString(metadata.userdata)}`);
/* Output:
Convert user data to string: OvenMediaEngine
*/
});
Player Initialization:
Call OvenPlayer.create()
function to create a player in the specified div.
Specify the stream type and URL in the sources
array. SEI is only supported in WebRTC streams.
Enable H.264 Network Abstraction Layer (NAL) parsing through parseStream.enabled: true
setting. This is required for SEI metadata processing.
SEI Data Processing:
Register a player.on('metaData', callback)
event listener to process SEI whenever it is received.
You can obtain the UUID, Timestamp, and custom data inserted by OvenMediaEngine from the event callback parameter.
metaData Event Callback Parameters
type
Always set to sei
, indicating this is SEI metadata
nalu
Uint8Array containing raw Network Abstraction Layer Unit (NALU) data of the SEI
sei
SEI parsing result containing the following sub-fields:
type
: SEI type
size
: Payload size
payload
: Raw SEI payload data (Uint8Array)
registered
Indicates whether the SEI was generated in the format defined by OvenMediaEngine. If true, the following additional fields are included
uuid
(when registered=true) Unique identifier inserted by OvenMediaEngine into the SEI
timecode
(when registered=true) Timestamp (milliseconds) when the SEI was inserted
userdata
(when registered=true) Uint8Array containing custom data. This data should be parsed according to the application's requirements
OvenPlayer version 0.10.39 and later can receive SEI data included in the video during WebRTC stream playback.
OvenPlayer can be downloaded from the OvenPlayer GitHub.
After extracting the downloaded file, copy all the files in the dist
directory to your project's library folder:
├─.github
├─demo
├─dist
│ ovenplayer.js
│ ovenplayer.js.map
│ RTCTransformWorker.worker.worker.js
│ RTCTransformWorker.worker.worker.js.map
├─docs
├─packages
└─src
Key files to copy:
ovenplayer.js
: OvenPlayer core library
RTCTransformWorker.worker.worker.js
: Worker script for WebRTC processing
Y
Contains the value of the event data.
<Values><SeiType>
N
Specifies SEI type.
Default: UserDataUnregistered
<Values><Data>
Y
Specifies custom data to be inserted into SEI.
It supports the macro ${EpochTime}
, which is replaced with the server's current epoch time in milliseconds when transmitted (e.g., 1747147513056).
<Values><KeyframeOnly>
N
Specifies the target for event insertion. If set to true
, the event will be inserted into the first keyframe after the specified interval has passed (Supported from version 0.18.2.0+).
Default: false