abhiTronix/vidgear

abhiTronix/vidgear

abhiTronix

Released VidGear – v0.1.6 (#85)

New Features:
  * **NetGear API:**
    * Added powerful ZMQ Authentication & Data Encryption features for NetGear API:
      * Added exclusive `secure_mode` param for enabling it.
      * Added support for two most powerful `Stonehouse` & `Ironhouse` ZMQ security mechanisms.
      * Added smart auth-certificates/key generation and validation features
    * Implemented Robust Multi-Server support for NetGear API:
      * Enables Multiple Servers messaging support with a single client.
      * Added exclusive `multiserver_mode` param for enabling it.
      * Added support for `REQ/REP` &  `PUB/SUB` patterns for this mode.
      * Added ability to send additional data of any datatype along with the frame in realtime in this mode.
    * Introducing exclusive Bi-Directional Mode for bidirectional data transmission:
      * Added new `return_data` parameter to `recv()` function.
      * Added new `bidirectional_mode` attribute for enabling this mode.
      * Added support for `PAIR` & `REQ/REP` patterns for this mode
      * Added support for sending data of any python datatype.
      * Added support for `message` parameter for non-exclusive primary modes for this mode
    * Implemented compression support with on-the-fly flexible frame encoding for the Server-end:
      * Added initial support for `JPEG`, `PNG` & `BMP` encoding formats 
      * Added exclusive options attribute `compression_format` & `compression_param` to tweak this feature
      * Client-end will now decode frame automatically based on the encoding as well as support decoding flags
    * Added `force_terminate` attribute flag for handling force socket termination at the Server-end if there's latency in the network. 
    * Implemented new *Publish/Subscribe(`zmq.PUB/zmq.SUB`)* pattern for seamless Live Streaming in NetGear API.

  * **PiGear API:**
    * Added new threaded internal timing function for PiGear to handle any hardware failures/frozen threads.
    * PiGear will not exit safely with `SystemError` if Picamera ribbon cable is pulled out to save resources.
    * Added support for new user-defined `HWFAILURE_TIMEOUT` options attribute to alter timeout.

  * **VideoGear API:** 
    * Added `framerate` global variable and removed redundant function.
    * Added `CROP_N_ZOOM` attribute in Videogear API for supporting Crop and Zoom stabilizer feature.

  * **WriteGear API:** Added new `execute_ffmpeg_cmd` function to pass a custom command to its FFmpeg pipeline.

  * **Stabilizer class:** 
    * Added new Crop and Zoom feature.
      * Added `crop_n_zoom` param for enabling this feature.
    * Updated docs.

  * **CLI & Tests updates:**
    * Replaced python 3.5 matrices with latest python 3.8 matrices in Linux environment.
    * Added full support for **Codecov** in all CLI environments.
    * Updated OpenCV to v4.2.0-pre(master branch). 
    * Added various Netgear API tests.
    * Added initial Screengear API test.
    * More test RTSP feeds added with better error handling in CamGear network test.
    * Added tests for ZMQ authentication certificate generation.
    * Added badge and Minor doc updates.

  * Added VidGear's official native support for MacOS environments.
    

Updates/Improvements:
  * Replace `print` logging commands with python's logging module completely.
  * Implemented encapsulation for class functions and variables on all gears.
  * Updated support for screen casting from multiple/all monitors in ScreenGear API.
  * Updated ScreenGear API to use *Threaded Queue Mode* by default, thereby removed redundant `THREADED_QUEUE_MODE` param.
  * Updated bash script path to download test dataset in `$TMPDIR` rather than `$HOME` directory for downloading testdata.
  * Implemented better error handling of colorspace in various videocapture APIs.
  * Updated bash scripts, Moved FFmpeg static binaries to `github.com`.
  * Updated bash scripts, Added additional flag to support un-secure apt sources.
  * CamGear API will now throw `RuntimeError` if source provided is invalid.
  * Updated threaded Queue mode in CamGear API for more robust performance.
  * Added new `camera_num` to support multiple Picameras.
  * Moved thread exceptions to the main thread and then re-raised.
  * Added alternate github mirror for FFmpeg static binaries auto-installation on windows oses.
  * Added `colorlog` python module for presentable colored logging.
  * Replaced `traceback` with `sys.exc_info`.
  * Overall APIs Code and Docs optimizations.
  * Updated Code Readability and Wiki Docs.
  * Updated ReadMe & Changelog with the latest changes.
  * Updated Travis CLI Tests with support for macOS environment.
  * Reformatted & implemented necessary MacOS related changes and dependencies in `travis.yml`.

Breaking Updates / Improvements / Changes
  * ⚠️ Python 2.7 legacy support dropped completely.
  * ⚠️ Source-code Relicensed to Apache 2.0 License.
  * Python 3+ are only supported legacies for installing Vidgear v0.1.6 and above.
  * Python 2.7 and 3.4 legacies support dropped from VidGear CLI tests.

Fixes
  * Reimplemented `Pub/Sub` pattern for smoother performance on various networks.
  * Fixed Assertion error in CamGear API during colorspace manipulation.
  * Fixed random freezing in `Secure Mode` and several related performance updates
  * Fixed `multiserver_mode` not working properly over some networks.
  * Fixed assigned Port address ignored bug (commit 073bca1).
  * Fixed several wrong definition bugs from NetGear API(commit 8f7153c).
  * Fixed unreliable dataset video URL(rehosted file on `github.com`).
  * Disabled `overwrite_cert` for client-end in NetGear API.
  * Disabled Universal Python wheel builds in `setup.cfg `file.
  * Removed duplicate code to import MSS(@BoboTiG) from ScreenGear API.
  * Eliminated unused redundant code blocks from library.
  * Fixed Code indentation in `setup.py` and updated new release information.
  * Fixed code definitions & Typos.
  * Fixed several bugs related to `secure_mode` & `multiserver_mode` Modes.
  * Fixed various macOS environment bugs.



Latest commit

62f32ad

Dec 31, 2019


VidGear Banner

VidGear is a powerful python Video Processing library built with multi-threaded Gears each with a unique set of trailblazing features. These APIs provides a easy-to-use, highly extensible, and multi-threaded wrapper around many underlying state-of-the-art libraries such as OpenCV ➶, FFmpeg ➶, picamera ➶, pafy ➶, pyzmq ➶ and python-mss ➶

The following functional block diagram clearly depicts the functioning of VidGear library:

@Vidgear Functional Block Diagram

TL;DR

Gears: What are these?

Installation

New-Release SneekPeak: v0.1.6

Documentation

For Developers/Contributors

Additional Info

What is vidgear?

“VidGear is an ultrafast➶, compact, flexible and easy-to-adapt complete Video Processing Python Library.”

What does it do?

“VidGear can read, write, process, send & receive video frames from various devices in real-time.”

What is its purpose?

“Built with simplicity in mind, VidGear lets programmers and software developers to easily integrate and perform complex Video Processing tasks in their existing or new applications, without going through various underlying library’s documentation and using just a few lines of code. Beneficial for both, if you’re new to programming with Python language or already a pro at it.”

For more advanced information, see the Wiki Documentation ➶.

VidGear is built with various Multi-Threaded APIs (a.k.a Gears) each with some unique function/mechanism.

Each of these API is designed exclusively to handle/control different device-specific video streams, network streams, and media encoders. These APIs provides an easy-to-use, highly extensible, and a multi-threaded wrapper around various underlying libraries to exploit their features and functions directly while providing robust error-handling.

These Gears can be classified as follows:

A. VideoCapture Gears:

  • CamGear: Targets various IP-USB-Cameras/Network-Streams/YouTube-Video-URL.
  • PiGear: Targets various Raspberry Pi Camera Modules.
  • ScreenGear: Enables ultra-fast Screen Casting.
  • VideoGear: A common API with Video Stabilizer wrapper.

B. VideoWriter Gear:

  • WriteGear: Handles easy Lossless Video Encoding and Compression.

C. Network Gear:

  • NetGear: Targets synchronous/asynchronous video frames transferring between interconnecting systems over the network.

CamGear

CamGear can grab ultra-fast frames from diverse range of devices/streams, which includes almost any IP/USB Cameras, multimedia video file format (upto 4k tested), various network stream protocols such as http(s), rtp, rstp, rtmp, mms, etc., plus support for live Gstreamer’s stream pipeline and YouTube video/livestreams URLs.

CamGear provides a flexible, high-level multi-threaded wrapper around OpenCV's VideoCapture class with access almost all of its available parameters and also employs pafy python APIs for live YouTube streaming. Furthermore, CamGear implements exclusively on Threaded Queue mode for ultra-fast, error-free and synchronized frame handling.

Following simplified functional block diagram depicts CamGear API’s generalized working:

CamGear Functional Block Diagram

CamGear API Guide:

>>> Usage Guide

VideoGear

VideoGear API provides a special internal wrapper around VidGear’s exclusive Video Stabilizer class.

Furthermore, VideoGear API can provide internal access to both CamGear and PiGear APIs separated by a special flag. Thereby, this API holds the exclusive power for any incoming VideoStream from any source, whether it is live or not, to access and stabilize it directly with minimum latency and memory requirements.

Below is a snapshot of a VideoGear Stabilizer in action:

VideoGear Stabilizer in action!


Original Video Courtesy @SIGGRAPH2013

Code to generate above VideoGear API Stabilized Video(See more detailed usage examples here):

# import libraries
from vidgear.gears import VideoGear
import numpy as np
import cv2

stream_stab = VideoGear(source='test.mp4', stabilize = True).start() # To open any valid video stream with `stabilize` flag set to True.
stream_org = VideoGear(source='test.mp4').start() # open same stream without stabilization for comparison

# infinite loop
while True:
  
  frame_stab = stream_stab.read()
  # read stabilized frames

  # check if frame is None
  if frame_stab is None:
    #if True break the infinite loop
    break
  
  #read original frame
  frame_org = stream_org.read()

  #concatenate both frames
  output_frame = np.concatenate((frame_org, frame_stab), axis=1)

  #put text
  cv2.putText(output_frame, "Before", (10, output_frame.shape[0] - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0,255,0), 2)
  cv2.putText(output_frame, "After", (output_frame.shape[1]//2+10, frame.shape[0] - 10),cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0,255,0), 2)
  
  cv2.imshow("Stabilized Frame", output_frame)
  # Show output window

  key = cv2.waitKey(1) & 0xFF
  # check for 'q' key-press
  if key == ord("q"):
    #if 'q' key-pressed break out
    break

cv2.destroyAllWindows()
# close output window
stream_org.stop()
stream_stab.stop()
# safely close video streams.

VideoGear API Guide:

>>> Usage Guide

PiGear

PiGear is similar to CamGear but made to support various Raspberry Pi Camera Modules (such as OmniVision OV5647 Camera Module and Sony IMX219 Camera Module).

PiGear provides a flexible multi-threaded wrapper around complete picamera python library to interface with these modules correctly, and also grants the ability to exploit its various features like brightness, saturation, sensor_mode, etc. effortlessly.

Best of all, PiGear API provides excellent Error-Handling with features like a threaded internal timer that keeps active track of any frozen threads and handles hardware failures/frozen threads robustly thereby will exit safely if any failure occurs. So now if you accidently pulled your camera module cable out when running PiGear API in your script, instead of going into possible kernel panic/frozen threads, API exit safely to save resources.

Following simplified functional block diagram depicts PiGear API:

PiGear Functional Block Diagram

PiGear API Guide:

>>> Usage Guide

ScreenGear

ScreenGear API act as Screen Recorder, that can grab frames from your monitor in real-time either by define an area on the computer screen or fullscreen at the expense of insignificant latency. It also provide seemless support for capturing frames from multiple monitors.

ScreenGear provides a high-level multi-threaded wrapper around python-mss python library API and also supports a easy and flexible direct internal parameter manipulation.

Below is a snapshot of a ScreenGear API in action:

ScreenGear in action!

Code to generate the above result:

# import libraries
from vidgear.gears import ScreenGear
import cv2

stream = ScreenGear().start()

# infinite loop
while True:
  
  frame = stream.read()
  # read frames

  # check if frame is None
  if frame is None:
    #if True break the infinite loop
    break
  
  cv2.imshow("Output Frame", frame)
  # Show output window

  key = cv2.waitKey(1) & 0xFF
  # check for 'q' key-press
  if key == ord("q"):
    #if 'q' key-pressed break out
    break

cv2.destroyAllWindows()
# close output window

stream.stop()
# safely close video stream.

ScreenGear API Guide:

>>> Usage Guide

WriteGear

WriteGear handles various powerful Writer Tools that provide us the freedom to do almost anything imagine with multimedia files.

WriteGear API provide a complete, flexible & robust wrapper around FFmpeg, a leading multimedia framework. With WriteGear, we can process real-time video frames into a lossless compressed format with any suitable specification in just few easy lines of codes. These specifications include setting any video/audio property such as bitrate, codec, framerate, resolution, subtitles, etc. easily as well complex tasks such as multiplexing video with audio in real-time(see this example wiki). Best of all, WriteGear grants the freedom to play with any FFmpeg parameter with its exclusive custom Command function(see this example wiki), while handling all errors robustly.

In addition to this, WriteGear also provides flexible access to OpenCV’s VideoWriter API which provides some basic tools for video frames encoding but without compression.

WriteGear primarily operates in the following two modes:

  • Compression Mode: In this mode, WriteGear utilizes FFmpeg's inbuilt encoders to encode lossless multimedia files. It provides us the ability to exploit almost any available parameters available within FFmpeg, with so much ease and flexibility and while doing that it robustly handles all errors/warnings quietly. You can find more about this mode here.

  • Non-Compression Mode: In this mode, WriteGear utilizes basic OpenCV’s inbuilt VideoWriter API. Similar to compression mode, WriteGear also supports all parameters manipulation available within OpenCV’s VideoWriter API. But this mode lacks the ability to manipulate encoding parameters and other important features like video compression, audio encoding, etc. You can learn about this mode here.

Following functional block diagram depicts WriteGear API’s generalized working:

WriteGear Functional Block Diagram

WriteGear API Guide:

>>> Usage Guide

NetGear

NetGear is exclusively designed to transfer video frames synchronously and asynchronously between interconnecting systems over the network in real-time.

NetGear implements a high-level wrapper around PyZmQ python library that contains python bindings for ZeroMQ – a high-performance asynchronous distributed messaging library that aim to be used in distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can run without a dedicated message broker.

NetGear provides seamless support for bidirectional data transmission between receiver(client) and sender(server) through bi-directional synchronous messaging patterns such as zmq.PAIR (ZMQ Pair Pattern) & zmq.REQ/zmq.REP (ZMQ Request/Reply Pattern).

NetGear also supports real-time frame Encoding/Decoding compression capabilities for optimizing performance while sending the frames directly over the network, by encoding the frame before sending it and decoding it on the client’s end automatically in real-time.

For security, NetGear implements easy access to ZeroMQ’s powerful, smart & secure Security Layers, that enables strong encryption on data, and unbreakable authentication between the Server and the Client with the help of custom certificates/keys and brings easy, standardized privacy and authentication for distributed systems over the network.

Best of all, NetGear can robustly handle Multiple Servers devices at once, thereby providing access to seamless Live Streaming of the multiple device in a network at the same time.

NetGear as of now seamlessly supports three ZeroMQ messaging patterns:

Following functional block diagram depicts generalized functioning of NetGear API:

NetGear Functional Block Diagram

NetGear API Guide:

>>> Usage Guide

  • ⚠️ Python 2.7 legacy support dropped in v0.1.6 !

  • NetGear API:

    • Added powerful ZMQ Authentication & Data Encryption features for NetGear API
    • Added robust Multi-Server support for NetGear API.
    • Added exclusive Bi-Directional Mode for bidirectional data transmission.
    • Added frame-compression support with on-the-fly flexible encoding/decoding.
    • Implemented new Publish/Subscribe(zmq.PUB/zmq.SUB) pattern for seamless Live Streaming in NetGear API.
  • PiGear API:

    • Added new threaded internal timing function for PiGear to handle any hardware failures/frozen threads
    • PiGear will not exit safely with SystemError if Picamera ribbon cable is pulled out to save resources.
  • WriteGear API: Added new execute_ffmpeg_cmd function to pass a custom command to its internal FFmpeg pipeline.

  • Stabilizer class: Added new Crop and Zoom feature.

  • Added VidGear’s official native support for MacOS environment and many more…

Prerequisites:

Before installing VidGear, you must verify that the following dependencies are met:

  • ⚠️ Must be using only supported Python legacies and also pip already installed and configured.

  • OpenCV: VidGear must require OpenCV(3.0+) python enabled binaries to be installed on your machine for its core functions. For its installation, you can follow these online tutorials for linux and raspberry pi, otherwise, install it via pip:

      pip3 install -U opencv-python       #or install opencv-contrib-python similarly
  • FFmpeg: VidGear must require FFmpeg for its powerful video compression and encoding capabilities. 🌟 Follow this FFmpeg wiki page for its installation. 🌟

  • picamera: Required if using Raspberry Pi Camera Modules(such as OmniVision OV5647 Camera Module) with your Raspberry Pi machine. You can easily install it via pip:

    Also, make sure to enable Raspberry Pi hardware-specific settings prior to using this library.

  • mss: Required for using Screen Casting. Install it via pip:

  • pyzmq: Required for transferring live video frames through ZeroMQ messaging system over the network. Install it via pip:

  • pafy: Required for direct YouTube Video streaming capabilities. Both pafy and latest only youtube-dl(as pafy’s backend) libraries must be installed via pip as follows:

      pip3 install pafy
      pip3 install -U youtube-dl

Available Installation Options:

Option 1: PyPI Install

Best option for quickly getting VidGear installed.

Option 2: Release Archive Download

Best option if you want a compressed archive.

VidGear releases are available for download as packages in the latest release.

Option 3: Clone the Repository

Best option for trying latest patches(maybe experimental), Pull Requests, or contributing to development.

You can clone this repository’s testing branch for development and thereby can install as follows:

 git clone https://github.com/abhiTronix/vidgear.git
 cd vidgear
 git checkout testing
 sudo pip3 install .

The full documentation for all VidGear classes and functions can be found in the link below:

  • Prerequisites: Testing VidGear require some additional dependencies & data which can be downloaded manually as follows:

    • Clone & Install Testing Branch

    • Download few additional python libraries:

       pip3 install six
       pip3 install pytest
    • Download Test Dataset: To perform tests, additional test dataset is required, which can be downloaded (to temp dir) by running bash script as follows:

       chmod +x scripts/bash/prepare_dataset.sh
       .scripts/bash/prepare_dataset.sh               #for Windows, use `sh scripts/bash/prepare_dataset.sh`
  • Run Tests: Then various VidGear tests can be run with pytest(in VidGear’s root folder) as below:

     pytest -sv                                   #-sv for verbose output.

See contributing.md.

  • Python 3+ are only supported legacies for installing Vidgear v0.1.6 and above.
  • ⚠️ Python 2.7 legacy support dropped in v0.1.6.

See changelog.md

Here is a Bibtex entry you can use to cite this project in a publication:

@misc{vidgear,
    Title = {vidgear},
    Author = {Abhishek Thakur},
    howpublished = {url{https://github.com/abhiTronix/vidgear}}   
  }

Copyright © abhiTronix 2019

This library is licensed under the Apache 2.0 License.

Other News

Menu
Need Help? Chat with us