JeVois  1.21
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
No Matches
Go to the documentation of this file.
1// ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
3// JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4// California (USC), and iLab at USC. See and for information about this project.
6// This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7// redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8// Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9// without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10// License for more details. You should have received a copy of the GNU General Public License along with this program;
11// if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
13// Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14// Tel: +1 213 740 3527 - - -
15// ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16/*! \file */
18#pragma once
20#include <mutex>
21#include <future>
22#include <deque>
23#include <atomic>
24#include <linux/usb/video.h> // for uvc_streaming_control
25#include <linux/videodev2.h>
30// for UVC gadget specific definitions; yes, this is only in the kernel tree, kernel maintainers should expose those
31// definitions in the standard headers instead:
32//#include "../../../../lichee/linux-3.4/drivers/usb/gadget/uvc.h"
33// We now include this file in our tree in so our code can compile without requiring the kernel source tree:
34#include "uvc.h"
36namespace jevois
38 class VideoInput;
39 class Engine;
40 class VideoBuffers;
42 //! JeVois gadget driver - exposes a uvcvideo interface to host computer connected over USB
43 /*! Gadget is a user-space interface to the Linux kernel's gadget driver implemented by JeVois. A USB gadget driver is
44 a USB driver that runs on a device, as opposed to a USB driver that runs on a host computer. Gadget drivers
45 implement USB functions such as USB printers, USB modems, or USB cameras. In JeVois, the Gadget driver makes the
46 JeVois hardware platform appear as a UVC (USB Video Class) camera. Because UVC has been standardized, no special
47 driver is needed on the host computer to start using the UVC device, as virtually all modern operating systems
48 already provide a built-in driver for UVC-compliant cameras.
50 The internal operation of Gadget is quite complex, especially due to its tight interaction with the kernel-side
51 UVC gadget driver. Yet, for users, Gadget is just another VideoOutput device: One can set its format, obtain blank
52 image buffers from it using get(), paint results into those buffers, and send them to the host computer when
53 complete using send().
55 Gadget implements a zero-copy, zero-wait access to output video frames, that is:
56 - the pixel data of the image you obtain via get() is directly the memory-mapped pixel buffer that the silicon
57 hardware on the JeVois chip will use via direct-memory-access (DMA) to stream the data out over the USB link;
58 - as soon as you call send() that buffer will be queued for sending over USB (as opposed to having a fixed,
59 regular interval at which images may be streamed out). Gadget has several image buffers, allowing one to be
60 streamed out over USB while another is being handed over for filling by your application via get(). These
61 buffers are recycled, i.e., once send() is called, the underlying buffer is streamed over USB and then sent back
62 to the Gadget for future access by your code.
64 Most programmers will never use Gadget directly, instead using Engine and OutputFrame. \ingroup core */
65 class Gadget : public VideoOutput
66 {
67 public:
68 //! Construct and open the device
69 /*! A vaid non-null camera is required for this gadget to work. To avoid testing for a non-null camera on each
70 operation of the gadget, we only test once at construction and then assume the camera will remain operational
71 for the lifetime of the gadget. Use 0 for nbufs to set it automatically. */
72 Gadget(std::string const & devname, VideoInput * camera, Engine * engine, size_t const nbufs = 0,
73 bool multicam = false);
75 //! Close the device and free all resources
76 virtual ~Gadget();
78 //! Set the video format and frame rate
79 void setFormat(jevois::VideoMapping const & m) override;
81 //! Get a pre-allocated image so that we can fill the pixel data and later send out over USB using send()
82 /*! May throw if not buffer is available, i.e., all have been queued to send to the host but have not yet been
83 sent. Application code must balance exactly one send() for each get(). */
84 void get(RawImage & img) override;
86 //! Send an image out over USB to the host computer
87 /*! May throw if the format is incorrect or std::overflow_error if we have not yet consumed the previous image. */
88 void send(RawImage const & img) override;
90 //! Start streaming
91 void streamOn() override;
93 //! Abort streaming
94 /*! This only cancels future get() and done() calls, one should still call streamOff() to turn off streaming. */
95 void abortStream() override;
97 //! Stop streaming
98 void streamOff() override;
100 private:
101 volatile int itsFd;
102 bool const itsMulticam;
103 size_t itsNbufs;
104 VideoBuffers * itsBuffers;
105 VideoInput * itsCamera;
106 Engine * itsEngine;
108 void run(); // Function to service requests, runs in a separate thread
109 std::future<void> itsRunFuture;
110 std::atomic<bool> itsRunning;
112 void processEvents();
113 void processEventSetup(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
114 void processEventStandard(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
115 void processEventClass(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
116 void processEventData(struct uvc_request_data & data);
117 void processEventControlData(struct uvc_request_data & data);
118 void processEventControl(uint8_t req, uint8_t cs, uint8_t entity_id, uint8_t len, struct uvc_request_data & resp);
119 void processEventStreaming(uint8_t req, uint8_t cs, struct uvc_request_data & resp);
120 void processVideo();
122 struct v4l2_format itsFormat;
123 float itsFps;
124 void fillStreamingControl(struct uvc_streaming_control * ctrl, VideoMapping const & m);
125 std::atomic<bool> itsStreaming;
126 int itsErrorCode;
127 int itsControl;
128 int itsEntity;
129 struct uvc_streaming_control itsProbe;
130 struct uvc_streaming_control itsCommit;
132 std::deque<RawImage> itsImageQueue;
133 std::deque<size_t> itsDoneImgs;
135 mutable std::timed_mutex itsMtx;
136 };
138} // namespace jevois
JeVois processing engine - gets images from camera sensor, processes them, and sends results over USB...
Definition Engine.H:414
JeVois gadget driver - exposes a uvcvideo interface to host computer connected over USB.
Definition Gadget.H:66
void streamOff() override
Stop streaming.
Definition Gadget.C:785
void abortStream() override
Abort streaming.
Definition Gadget.C:777
void get(RawImage &img) override
Get a pre-allocated image so that we can fill the pixel data and later send out over USB using send()
Definition Gadget.C:813
virtual ~Gadget()
Close the device and free all resources.
Definition Gadget.C:185
void setFormat(jevois::VideoMapping const &m) override
Set the video format and frame rate.
Definition Gadget.C:201
void streamOn() override
Start streaming.
Definition Gadget.C:721
void send(RawImage const &img) override
Send an image out over USB to the host computer.
Definition Gadget.C:856
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition RawImage.H:111
Collection of buffers for V4L2 video frames (Camera or Gadget) with hooks to the MMAP'd areas.
Base class for video input, which will get derived into Camera and MovieInput.
Definition VideoInput.H:32
Base class for video output. Gadget, MovieOutput, VideoDisplay, and VideoOutputNone derive from it.
Definition VideoOutput.H:28
Main namespace for all JeVois classes and functions.
Definition Concepts.dox:2
Simple struct to hold video mapping definitions for the processing Engine.