Qt - H.264 video streaming using FFmpeg libraries -

i trying ip camera stream in qt widget application. first, connect udp port of ip camera. ip camera streaming h.264 encoded video. after socket bind, on each readyread() signal filling buffer received datagrams in order full frame.

variable initialization:

avcodec *codec; avcodeccontext *codecctx; avframe *frame; avpacket packet; this->buffer.clear(); this->socket = new qudpsocket(this);  qobject::connect(this->socket, &qudpsocket::connected, this, &h264videostreamer::connected); qobject::connect(this->socket, &qudpsocket::disconnected, this, &h264videostreamer::disconnected); qobject::connect(this->socket, &qudpsocket::readyread, this, &h264videostreamer::readyread); qobject::connect(this->socket, &qudpsocket::hostfound, this, &h264videostreamer::hostfound); qobject::connect(this->socket, signal(error(qabstractsocket::socketerror)), this, slot(error(qabstractsocket::socketerror))); qobject::connect(this->socket, &qudpsocket::statechanged, this, &h264videostreamer::statechanged);  avcodec_register_all();  codec = avcodec_find_decoder(av_codec_id_h264); if (!codec){    qdebug() << "codec not found";    return; }  codecctx = avcodec_alloc_context3(codec); if (!codecctx){     qdebug() << "could not allocate video codec context";     return; }  if (codec->capabilities & codec_cap_truncated)       codecctx->flags |= codec_flag_truncated;  codecctx->flags2 |= codec_flag2_chunks;  avdictionary *dictionary = nullptr;  if (avcodec_open2(codecctx, codec, &dictionary) < 0) {     qdebug() << "could not open codec";     return; } 

algorithm follows:

void h264videoimageprovider::readyread() { qbytearray datagram; datagram.resize(this->socket->pendingdatagramsize()); qhostaddress sender; quint16 senderport;  this->socket->readdatagram(datagram.data(), datagram.size(), &sender, &senderport);  qbytearray rtpheader = datagram.left(12); datagram.remove(0, 12);  int nal_unit_type = datagram[0] & 0x1f; bool start = (datagram[1] & 0x80) != 0;  int seqno = rtpheader[3] & 0xff;  qdebug() << "h264 video decoder::readyread()"          << "from: " << sender.tostring() << ":" << qstring::number(senderport)          << "\n\tdatagram size: " << qstring::number(datagram.size())          << "\n\th264 rtp header (hex): " << rtpheader.tohex()          << "\n\th264 video data (hex): " << datagram.tohex();  qdebug() << "nal_unit_type = " << nal_unit_type << " - " << getnalunittypestr(nal_unit_type); if (start)     qdebug() << "start";  if (nal_unit_type == 7){     this->sps = datagram;     qdebug() << "sequence parameter found = " << this->sps.tohex();     return; } else if (nal_unit_type == 8){     this->pps = datagram;     qdebug() << "picture parameter found = " << this->pps.tohex();     return; }  //video_frame if (start){     if (!this->buffer.isempty())         decodebuf();      this->buffer.clear();     qdebug() << "initializing new buffer...";      this->buffer.append(char(0x00));     this->buffer.append(char(0x00));     this->buffer.append(char(0x00));     this->buffer.append(char(0x01));      this->buffer.append(this->sps);      this->buffer.append(char(0x00));     this->buffer.append(char(0x00));     this->buffer.append(char(0x00));     this->buffer.append(char(0x01));      this->buffer.append(this->pps);      this->buffer.append(char(0x00));     this->buffer.append(char(0x00));     this->buffer.append(char(0x00));     this->buffer.append(char(0x01)); }  qdebug() << "appending buffer data..."; this->buffer.append(datagram); } 

• first 12 bytes of datagram rtp header
• everything other video data
• last 5 bits of first video data byte, says nal unit type is. 1 of following 4 values (1 - coded non-idr slice, 5 code idr slice, 7 sps, 8 pps)
• 5th bit in 2nd video data byte says if datagram start data in frame
• all video data stored in buffer starting start
• once new frame arrives - start set, decoded , new buffer generated

• frame decoding generated this:

  00 00 00 01

  sps

  00 00 00 01

  pps

  00 00 00 01

  concatenated video data

• decoding made using avcodec_decode_video2() function ffmpeg library

  void h264videostreamer::decode() {  av_init_packet(&packet);  av_new_packet(&packet, this->buffer.size()); memcpy(packet.data, this->buffer.data_ptr(), this->buffer.size()); packet.size = this->buffer.size();      frame = av_frame_alloc(); if(!frame){     qdebug() << "could not allocate video frame";     return; }  int got_frame = 1;  int len = avcodec_decode_video2(codecctx, frame, &got_frame, &packet);  if (len < 0){     qdebug() << "error while encoding frame.";     return; }  //if(got_frame > 0){ // got_frame 0 //    qdebug() << "data decoded: " << frame->data[0]; //}  char * framedata = (char *) frame->data[0]; qbytearray decodedframe; decodedframe.setrawdata(framedata, len);  qdebug() << "data decoded: " << decodedframe;  av_frame_unref(frame); av_free_packet(&packet);  emit imagereceived(decodedframe); } 

my idea in ui thread receives imagereceived signal, convert decodedframe directly in qimage , refresh once new frame decoded , sent ui.

is approach decoding h.264 stream? facing following problems:

• avcodec_decode_video2() returns value same encoded buffer size. possible encoded , decoded date same size?
• got_frame 0, means never received full frame in result. can reason? video frame incorrectly created? or video frame incorrectly converted qbytearray avframe?
• how can convert decoded avframe qbytearray, , can converted qimage?

the whole process of manually rendering frames can left library. if purpose qt gui live feed ip camera can use libvlc library. can find example here: https://wiki.videolan.org/libvlc_samplecode_qt