SentiSight SDK questions

[Martynas]

Hi,

does your "F:\fyp\done\List\background.seil" file contains lines separated with empty rows like below:

Code: Select all

Background/Background-001.bmp
//empty row
Background/Background-002.bmp
//empty row
Background/Background-003.bmp
//empty row
Background/Background-004.bmp


If yes then there is a possibility that your application is trying to load the file with empty name. Please check what file you are trying to access before you get this error.

[hhanyuan]

no.. my seil file look like this

Code: Select all

Background\Background 0000.jpg
Background\Background 0001.jpg
Background\Background 0002.jpg
Background\Background 0003.jpg
Background\Background 0004.jpg
Background\Background 0005.jpg
Background\Background 0006.jpg
Background\Background 0007.jpg
Background\Background 0008.jpg
Background\Background 0009.jpg

is it that i can not (environment.new line)or additional codes need to be added??

Regards,
Alvin

[Martynas]

What is the full path to "Background 0000.jpg"?

[hhanyuan]

hi,

The path will be"F:\FYP\done\Background".
I'm sincerely sorry that i had bother you so much..

regards,
Alvin

[Martynas]

Please, find the file "Background 0000.jpg" on your computer and copy the full path to it and paste it here.

[hhanyuan]

So sorry,

i have solve the previous problem due to different name in seil file so unable to load.
for Recognise wise, how do i actually program the camera to take video :by Nvideowriter
as i saw from the documentation and the tutorial, there's only reader and writer to use

[Martynas]

Hello,

If you need to write a video file captured from the camera, then you need to get frames from camera, by using Camera.GetCurrentFrame() and pass captured NImage to NVideoWritter, by using NVideoWritter.WriteFrame(NImage).

[hhanyuan]

Hello,
if i is to just use photo for recognition is it possible,
if possible, is it just use the same programming technique to program or other sources.
if not possible, is it that only video can be used in the recognization.

thanks and regards,
alvin

[Martynas]

Hello,

it is possible to use images, cameras and video files with the SentiSight.

[hhanyuan]

Hi,
the code error below have a code error of -7

Code: Select all

private void timer1_Tick(object sender, EventArgs e)
        {
            int intwidth, intheight;
            double dblframerate;
            intwidth=150;
            intheight=150;
            dblframerate = 1;
            String videofile = @"D:\FYP\done\video\video.avi";
            String background = @"D:\FYP\done\List\background.seil";
            NVideoWriterOptions option= null;
            NVideoWriter framewriter=new NVideoWriter(videofile,intwidth,intheight,dblframerate,option);
            foreach (Camera camera in cameraMan.Cameras)
            {
               if (i != 10)
                {
                 
               
                NImage testImg = (NImage)camera.GetCurrentFrame().Clone();
                NImage testvid = (NImage)camera.GetCurrentFrame().Clone();
                framewriter.WriteFrame((NImage)camera.GetCurrentFrame().Clone());

               
                pictureBox.Image = testImg.ToBitmap();
                string name = String.Format("Background {1:d4}.jpg", camera.ID, i);
               
                testImg.Save(@"D:\FYP\done\Image\Background\" + name);
                File.AppendAllText(background, name + Environment.NewLine);
                i++;
                Thread.Sleep(500);
                listBox1.Items.Add(name);
                   testImg.Dispose();
                testvid.Dispose();
                 framewriter.Dispose();
                 
               }
               
               
                else
                   {
                    camera.StopCapturing();
                    timer1.Enabled = false;
                    i = 0;
               
               
            }
            }
         
        }

        private void button2_Click(object sender, EventArgs e)
        {

            foreach (Camera camera in cameraMan.Cameras)
            {
                camera.StartCapturing();
                timer2.Enabled = true;
                button2.Enabled = false;
            }

        }

please help me out with this problem..

thanks,
alvin

[Martynas]

Hello,

please, send your solution to support@neurotechnology.com with the detailed description of the issue.

[volx_jeff]

Hi all

I'm new at SentiSight. Is there any document on "how SentiSight" works?

[Sergey]

Hi all

I'm new at SentiSight. Is there any document on "how SentiSight" works?

Have you read the SentiSight object learning and recognition processes?
Also you can download SentiSight algorithm demo to try the SentiSight and see how it works.

[mohsinhasan]

Hi,
I'm new to sentisight sdk and was referring to this forum for my project work,which includes a win form which will show a camera on picture box and will capture frames on timertick event,which I'm able to achieve but how can i integrate the Learn and recognition feature in my project.A brief information would suffice me..Help me out in achieving further goals.
Thank you

[Martynas]

Hello,

learning and recognition tutorials are included in the SentiSight SDK tutorials folder.
Also it is useful to read the documentation of SentiSight to get the main ideas of the learning and recognition. Documentation is located in "documentation" folder, included in the SDK.

[mohsinhasan]

HI,
Thank you for your quick response..I'm going through the documentation and tutorials but in my winform application I'm getting this error{An unhandeled win32 exception occured in objectDetection.vshost.exe}whwn i debugged it I came to know about the error which is CameraMan.dll cannot be found,but i have all the wrappers in my bin directory as well as refrenced.
I'm not able to display my webcam in picturebox on timertick event as guided by you.
Thank you

[Martynas]

Hello,

SentiSight SDK has two types of dlls. .Net dlls and C/C++ dlls. .Net dlls depend on C/C++ dlls. So if you have not installed the SentiSight SDK, then path to dlls are not in System variable PATH and your application can not find them.
When you add references in your application, VS usually copies them to your application output folder, but it does not copy all other dependent dlls. Solution int his case would be to copy all dlls from the SentiSight SDK to your application output folder.
If the application is not detecting the camera, then usually the issue is that the application can not find "Cmm" folder in folder from where the NDeviceManager.dll is loaded. Solution in this case would be to copy the Cmm folder to the folder from where NDeviceManager.dll is loaded.

[mohsinhasan]

Hi,
Thank you again for your response,
I'm able to solve the problem regarding Dll and on form load in my picturebox only black screen is shown,I'm not able to see the live camera video..
and also I'm not able to find any Cmm folder..

[mohsinhasan]

Hi,I got it there was problem with SDK,i downloaded SDK v2.0,so I have downloaded the SentiSight SDK v2.1 and will work on the steps suggested by you.

Thank you

[mohsinhasan]

Hi,
I downloaded the SDk2.1 and currently working on that,now I have a task to do, that is how can i show two objects simultaneously,for eg:-suppose there is one 3d animated object and other a still object like book,so after learning will it be able to identify both the objects.How can i achieve object overlapping during the video capture??

I hope you got my query..
Thank You

[Martynas]

Hi,

please specify your task in details.
As I understand you want to show an animated object and a still object in the same video. Am I right? SentiSight does not provide this functionality.

[mohsinhasan]

Hi,
Thank you for the response.
What i want is that i have a 3d object that sentisight will learn and during the live recognition it should detect that 3d object.
It doesnt matter if there be 2 objects,just a 3D object should be recognized during the live recognition.

To be more specific I'm using a marker based thing to show that 3D object during the video,and i want this to integrate with sentisight,that it should learn and once it detects the marker and the 3D object during the learn process,so that during recognition it should show my 3d object.
Note:-The 3D object is not animated.

Thank you

[mohsinhasan]

Hi,
I have achieved what i was doing, now I have a new Task first tell me is there a possibility that after the object is detected during live recognition on that object can i show a 3D object on that recognized object..Is there a possibility?
Tell me how can i achieve it?

Thank You

[Martynas]

Hello,

please explain everything in details, if possible with example.

[mohsinhasan]

Hi,
Suppose during camera motion I have an object Book and it was learned by the application,during live recognition it will detect the object via Identifier that it is a Book,what i want to do is on that book can i show my 3D object during camera motion..That book should work as a marker and whenever that book is detected it will show a 3D object on it.

[Martynas]

Hello,

as you can see in the samples provided with SentiSight SDK, the object is selected in the video frame during recognition. So you can use the same method to show your 3D object.

[mohsinhasan]

HI,
Can you help me with the coding?Which method as to where i can show it?
Please

[Martynas]

Hello,

sorry, but i can not write the code instead of you, but if you will look into the SentiSight sample SentiSightSampleCS you will find ObjectSelectionBox.cs source code where you will find OnPaint method. In this method you will see how the Selection is drawed. It uses ObjectSelections to draw selection box. This object contains Neurotec.SentiSight.SEShape object which has coordinates of center of the shape. So you can use these coordinates to draw you 3D object.

[mohsinhasan]

Hi,

Thank you so much for helping..i'll work into that and if I have problems i ll update you..
Thank you so much once again

[surjer]

Hello, After downloading the SentiSight 2.1 SDK I cant find the 64bit versions of the .NET wrappers or anything 64bit related for that matter. Any ideas where I should be looking? Its possible I am not looking in the correct places.

Thanks in advance,

Jerry

Admin: merged to "SentiSight SDK questions"

[vaidasz]

Hello, After downloading the SentiSight 2.1 SDK I cant find the 64bit versions of the .NET wrappers or anything 64bit related for that matter. Any ideas where I should be looking? Its possible I am not looking in the correct places.

Thanks in advance,

Jerry

Hello Jerry,

SentiSight is not supported under 64-bit OSes at the moment.

[Guy]

I'm using the demo app from 2.1. How do I use automatic learning from Camera. I select Data Source: Camera, click Learn. I want to capture background, then holder, then holder + object, then save model. This does not work for me? Thanks.

[vaidasz]

I'm using the demo app from 2.1. How do I use automatic learning from Camera. I select Data Source: Camera, click Learn. I want to capture background, then holder, then holder + object, then save model. This does not work for me? Thanks.

Hello Jerry,

The tutorials and samples are provided with SDK. Please check them.

[ronenvvv]

Hi,

I stared POC the API maybe use it in my application, I'm using the latest trail from the site with .net 3.5.

I have few questions:

1. after using the learn, creating the model and saving it to the DB file I want to show the drawn area from the image (set it on a new picturebox object). how can I get the image/bitmap from the Neurotec API (I didnt see any get for it, the learn gets the original image with the shapes and masks). maybe giving the original image and the shapes and getting back the drawn image.

2. do you have any kind of benchmark for the learning and recognizing? I need to do some load and stress testing for this POC

Thanks!
Ronen

Admin: merged to "SentiSight SDK questions"

[vaidasz]

Hi,

I stared POC the API maybe use it in my application, I'm using the latest trail from the site with .net 3.5.

I have few questions:

1. after using the learn, creating the model and saving it to the DB file I want to show the drawn area from the image (set it on a new picturebox object). how can I get the image/bitmap from the Neurotec API (I didnt see any get for it, the learn gets the original image with the shapes and masks). maybe giving the original image and the shapes and getting back the drawn image.

2. do you have any kind of benchmark for the learning and recognizing? I need to do some load and stress testing for this POC

Thanks!
Ronen

Hello,

Here are the answers to your questions:
1. No, it is not possible to do that using SentiSight SDK but you can implement it by yourself.
2. No, we do not have any benchmark.

[greg]

Hello.
I'm making evaluation of the SentiSight capabilities (as well as future support) for my company, and after few days of working with demo I've understood that I'll need to clear few things.
1) .NET wrapping: There is only .NET 2.0 support now - will it be extended to next versions? When?
2) Is there some benchmarking of the recognition\learning processes performance - say results of speed of recognition for models containing many submodels? with both High and Low Speeds? on different processors? does the kind of video card plays?
Is type of image loaded to model playing role in performance rate?
3) What are exactly image formats the SentiSight supports? I noticed that TIFF format bitmaps fails, for example. Is there functionality to check the bitmap before it loaded - if its format fits? Will it be extended to all .NET System.Drawing.Imaging.ImageFormats? When?
4) The SEShape has some hidden functionality inside not described very well in the SDK manual - for example property IsValid - does it check Convexity of the shape? What does the Center property?
5) If I put treshold of the engine recognition to 0 - does it mean that all recognition models will work? With what similarity score?
In short can I build some model to get list of all models recognized sorted by similarity score?
Thank you in advance,
Greg.

[vaidasz]

Hello Greg,

Here are the answers to your questions:

1) .NET wrapping: There is only .NET 2.0 support now - will it be extended to next versions? When?

No, we do not have such plans in near future. But as I know there shouldn't be any issues using .NET 2.0 with .NET 4.0.

2) Is there some benchmarking of the recognition\learning processes performance - say results of speed of recognition for models containing many submodels? with both High and Low Speeds? on different processors? does the kind of video card plays?

The only test of speed was performed gave the following results (tested on i7-920 processor):
Learning of 1 image takes 0.1261 sec.
There were used 36 images for 1 object - 4.5409 sec.
Matching speed 395 images/second
Test was performed using fast speed. Low speed is not predictable - it could be 2 or 3 times slower.
We have never performed tests with video cards.

Is type of image loaded to model playing role in performance rate?

If the image resolution is different the speed could decrease till square progression.
The bigger number of images leads to linear speed decrease.

3) What are exactly image formats the SentiSight supports? I noticed that TIFF format bitmaps fails, for example. Is there functionality to check the bitmap before it loaded - if its format fits? Will it be extended to all .NET System.Drawing.Imaging.ImageFormats? When?

SentiSight supports BMP, GIF, Nist IHead, JPEG, Lossless JPEG, JPEG 2000, PNG, TIFF and WSQ. You can check it on documentation provided with SDK (section "4.4.2 Image Format").
What regards TIFF image, could you please send it to me to check it?
To check image you can try to load from image and if you get NotSupportedException then this image is not supported.

4) The SEShape has some hidden functionality inside not described very well in the SDK manual - for example property IsValid - does it check Convexity of the shape? What does the Center property?

We know that documentation is our weak part but we are working on that.
SEShapeIsValid returns if shape is self intersected.
Center property returns the geometric center of the object.

5) If I put treshold of the engine recognition to 0 - does it mean that all recognition models will work? With what similarity score?

If you set threshold to 0 it will return all objects that are a little bit similar to the object you want (it should pass internal thresholds) and then you will have to check results by yourself (or you can use the one with the biggest score).

In short can I build some model to get list of all models recognized sorted by similarity score?

Yes, you can.

[alomaGurbcomb]

Is there any chance that the Delphi examples will be fixed?
Of course it is a bit difficult to evaluate a SDK when the examples are buggy...
Also I noticed some memory leaks in the Delphi examples. Memory not 'freed'.

[Martynas]

Hello,

about which SDK you are talking? Could you provide details on the SDK you are using and issues you are receiving?
SentiSight SDK does not support Delphi.

[Guy]

Hello, using your manual demo app, the program only recognizes 1 instance of the same object. Having 2 objects of the same kind, the recognition seems to alternate between them. How do I have both objects recognized & outlined at the same time?

[vaidasz]

Hello, using your manual demo app, the program only recognizes 1 instance of the same object. Having 2 objects of the same kind, the recognition seems to alternate between them. How do I have both objects recognized & outlined at the same time?

Hello,

Please change the speed to low and all objects will be found. Low speed does not mean that more accurate matching will be - it means that will search for all objects in the image.

[nikunj]

Hi, Martynas,

Actually, I am working on Object Recognition using SentiSight SDK.
In documentation, it's written that we have to provide Mask Image in which white pixels show the presence of object. That I understood.

but, if i took 20 images of object at different poses and different backgrounds then, in that case how to produce mask images of each image, should i make it manually through Paintbrush software ?

What i interpret with documentation is that, mask images are binary images containing 0 and 1 in which pixels representing specified object are having value 1 and others 0.

So, please suggest me some method to make mask images ..

Thank You

with regards,
Nikunj Chaudhary

Admin: merged to "SentiSight SDK questions"

[Martynas]

Hi, Martynas,

Actually, I am working on Object Recognition using SentiSight SDK.
In documentation, it's written that we have to provide Mask Image in which white pixels show the presence of object. That I understood.

but, if i took 20 images of object at different poses and different backgrounds then, in that case how to produce mask images of each image, should i make it manually through Paintbrush software ?

What i interpret with documentation is that, mask images are binary images containing 0 and 1 in which pixels representing specified object are having value 1 and others 0.

So, please suggest me some method to make mask images ..

Thank You

with regards,
Nikunj Chaudhary

Hello Nikunj,

if you would have a same background, then it would be possible to use the automatic foreground/background separation, as it is described in the documentation. In your situation, whenthe background is not same, then you can use polygon (shape) – to mark image region occupied by simpler objects. The polygon is specified by the shape argument of SELrnAddToModelEx function. See the SEShapeCreate function for details about the shape. If the mask is specified, the shape is not used in the learning process. It will than only used in the recognition stage to report the region the recognized object occupies on the recognition image.
So by using polygons (shapes) you can mark the learning object as it is done in our provided samples.

[parameshg]

Hi, Martynas,

Actually, I am working on Object Recognition using SentiSight SDK.
In documentation, it's written that we have to provide Mask Image in which white pixels show the presence of object. That I understood.

but, if i took 20 images of object at different poses and different backgrounds then, in that case how to produce mask images of each image, should i make it manually through Paintbrush software ?

What i interpret with documentation is that, mask images are binary images containing 0 and 1 in which pixels representing specified object are having value 1 and others 0.

So, please suggest me some method to make mask images ..

Thank You

with regards,
Nikunj Chaudhary

Hello Nikunj,

if you would have a same background, then it would be possible to use the automatic foreground/background separation, as it is described in the documentation. In your situation, whenthe background is not same, then you can use polygon (shape) – to mark image region occupied by simpler objects. The polygon is specified by the shape argument of SELrnAddToModelEx function. See the SEShapeCreate function for details about the shape. If the mask is specified, the shape is not used in the learning process. It will than only used in the recognition stage to report the region the recognized object occupies on the recognition image.
So by using polygons (shapes) you can mark the learning object as it is done in our provided samples.

Hi,

When using the SentiSight sample application, setting the learning mode to shape and drawing the outline, the application does not seems to recognize the object. Whereas, when the same object, using the same procedure, when the learning mode to set to blob, the application seems to work fine. Is there anything that I'm missing?

Thanks,
Param

[Martynas]

Hi,

could you provide images, which you have used in learning, so we would be able to comment the situation. You can send the email to support@neurotechnology.com with images and detailed description of the issue.

[parameshg]

Hi,

could you provide images, which you have used in learning, so we would be able to comment the situation. You can send the email to support@neurotechnology.com with images and detailed description of the issue.

Hey,

Thanks for the reply. We're using live video feed from a web cam and regular day-to-day objects like bottles, jars, etc. Any recommendations?

Thanks,
Param

[Martynas]

We're using live video feed from a web cam and regular day-to-day objects like bottles, jars, etc. Any recommendations?

we would be able to provide comments if the objects can be used with the SentiSight if we could see the images of objects.

[parameshg]

Hi,

I get an exception at the following line of code.

SEEngine.Recognition.Recognize(NImage)

Any suggestion would be helpful. Thanks.

System Spec:
Intel Xeon E5645 2.4 GHz 16 GB RAM
Windows Server 2008 x64 R2 Standard + SP1

Problem signature:
Problem Event Name: APPCRASH
Application Name: Try.exe
Application Version: 1.12.326.0
Application Timestamp: 4fb3c3c1
Fault Module Name: SentiSight.DLL
Fault Module Version: 3.0.0.0
Fault Module Timestamp: 4e7e296e
Exception Code: c0000005
Exception Offset: 000000000000828e
OS Version: 6.1.7601.2.1.0.272.7
Locale ID: 1033
Additional Information 1: 442a
Additional Information 2: 442aad694418854108a1d099a9fd029d
Additional Information 3: 224e
Additional Information 4: 224ebc1e515abdc94cfcbd067582492b

[vaidasz]

Hi,

I get an exception at the following line of code.

SEEngine.Recognition.Recognize(NImage)

Any suggestion would be helpful. Thanks.

System Spec:
Intel Xeon E5645 2.4 GHz 16 GB RAM
Windows Server 2008 x64 R2 Standard + SP1

Problem signature:
Problem Event Name: APPCRASH
Application Name: Try.exe
Application Version: 1.12.326.0
Application Timestamp: 4fb3c3c1
Fault Module Name: SentiSight.DLL
Fault Module Version: 3.0.0.0
Fault Module Timestamp: 4e7e296e
Exception Code: c0000005
Exception Offset: 000000000000828e
OS Version: 6.1.7601.2.1.0.272.7
Locale ID: 1033
Additional Information 1: 442a
Additional Information 2: 442aad694418854108a1d099a9fd029d
Additional Information 3: 224e
Additional Information 4: 224ebc1e515abdc94cfcbd067582492b

Hello,

Please send this issue to support@neurotechnology.com
In the e-mail describe the issue you are facing. Also provide step by step scenario you are performing. Specify what parameters do you use (also send the images you use). Is it your application?

[patiljagdish]

Hello,
I am new to sentisight and sorry for asking this question.
How to read and display from camera ?
Are there any simple tutorials to learn from scratch ?

Admin: merged to "SentiSight SDK questions"