Page 1 of 3

CT/MR scans > 3D viewer

Posted: Mon May 05, 2014 7:35 pm
by marc_256
A result of my post in the "off topic" section.
http://www.purebasic.fr/english/viewtop ... 17&t=59197

My project:

1) Converter to enlarge the gray-scale images size and contrast.
2) Convert this 2D images to a 3D mesh
3) Build a 2D CT/MT images to 3D Mesh viewer to see/detect early stage diseases.

next step is to read and convert directly from DICOM format to bmp gray-scale images.

some first results
First results - http://www.marc-systems.be/scan_view_001.png
Last results - http://www.marc-systems.be/new_2010/Vie ... amples.htm

marc,

Every help is welcome...

Re: CT/MR scans > 3D viewer

Posted: Mon May 05, 2014 10:25 pm
by Mythros
If this works for any 2d image COMPLETELY, im talking textures as well, then count me in! :D

Re: CT/MR scans > 3D viewer

Posted: Mon May 05, 2014 10:39 pm
by mpz
Hi Marc_256,

is it possible to load the "150 gray-scale images of (518x518) = 40 k Points" to a server (http://uploaded.net)? I will test to convert these pictures to a 40k points mesh

like the following mesh (= 100k Pointmesh). I thing this ist not difficult to realize

http://www.purebasic.fr/english/viewtop ... 00#p433109

nest step ist to makes moves with the camera and export a 3d film for 3d TVs or change grey color to color in realtime ...

Greetings Michael

Re: CT/MR scans > 3D viewer

Posted: Mon May 05, 2014 11:15 pm
by marc_256
Hi Mythros and mpz,

thanks for the help,
yes, I gone a need a lot of stuff ...

@mpz
I go try to make some convertions ... (10 images to start with)

The difficulty I have now is,
that the scans are made every 3.1mm distance,
and if I zoom in and out, I get some moire effect.

Image

* For now, the over-night version of my 3D viewer (test is the skeleton selection algorithm)
so from the 150 - 2D CT scan images, I can now select the skeleton.
Here you can see my mom almost naked (80+). :wink:

Image

marc,

Re: CT/MR scans > 3D viewer

Posted: Mon May 05, 2014 11:19 pm
by marc_256
@mpz,

I think you made a calculation error here,

518 px * 518 px * 150 images = 40 248 600 pixels/mesh
this is over the 40 million pixels

marc

Re: CT/MR scans > 3D viewer

Posted: Mon May 05, 2014 11:52 pm
by Mythros
Can you release the source code, marc? :) I would like to mess with it myself. Is there a way you can create triangles for the point cloud, based on the color of each pixel? For instance, you have:

Code: Select all

 .
. .

=
   |
  / \
 /   \
/_____\
Each color uses a heightmap-like algorithm in order to grab the correct height for each of the points in the pointcloud and thus allows for a TRUE 3D mesh.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 12:18 am
by marc_256
@Mythros,

I'm looking for it,
and found http://en.wikipedia.org/wiki/MeshLab

But it looks very difficult to do ??

The source code is very messy for now...need to clean up first.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 2:59 am
by Mythros
We should be able to just turn the image into a point cloud, then use the colors of each pixel to establish a "height" for the 3D mesh. Then, for the depth, use the light or color (not sure which) of either the 3d mesh or the color of each point in order to connect each point to each other point, forming triangles.

We basically just need to give each 2d image to 3d model its own depth in a correct sense. Basically, make it pop out with not just the points, but curves too. Instead of creating a "spikey" 3d object, it would give the correct depth for each 3d model created.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 6:21 am
by marc_256
Mythros wrote:We should be able to just turn the image into a point cloud, then use the colors of each pixel to establish a "height" for the 3D mesh. Then, for the depth, use the light or color (not sure which) of either the 3d mesh or the color of each point in order to connect each point to each other point, forming triangles.

We basically just need to give each 2d image to 3d model its own depth in a correct sense. Basically, make it pop out with not just the points, but curves too. Instead of creating a "spikey" 3d object, it would give the correct depth for each 3d model created.
Thats what I did already, Step 1 => I convert the 2D images to a point cloud.
But if you will connect all the same colored points, there will be >100000 of triangles...within the same color.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 10:48 am
by IdeasVacuum
I think having a high density of points might be enough, without meshing?

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 12:39 pm
by marc_256
IdeasVacuum wrote:I think having a high density of points might be enough, without meshing?
Yes, you are right, but finding high resolution CT scans is the problem here.
The ones I have are 518x518 px for +-500mm => resolution => +-1mm/px
and are 3.1 mm distance between each scan.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 2:55 pm
by Mythros
It doesn't matter the amount of triangles, just as long as all the triangles are interconnected, then we can have an actual 3D mesh. And we don't connect the same colored triangles. We connect all triangles to each other.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 3:46 pm
by IdeasVacuum
Mythos, the number of mesh faces does matter, it matters a lot, because viewing performance can be severely hampered. This is a known issue in CAD/CAM apps for example.

Marc, 1mm/px is not too bad? The main issue is the 3.1mm distance between slices. I think it is possible to vary this on the machine itself, but 3.1mm is probably already the minimum distance. The more slices, the longer it takes and that has two issues - firstly, being scanned is stressful for the patient and secondly there are not enough machines to meet demand so the fastest possible scan session is always a requirement.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 8:12 pm
by marc_256
Mythros wrote:It doesn't matter the amount of triangles, just as long as all the triangles are interconnected, then we can have an actual 3D mesh. And we don't connect the same colored triangles. We connect all triangles to each other.
Like I wrote here,

518 px * 518 px * 150 images = 40 248 600 pixels/mesh
this is over the 40 million pixels

If you want to connect all the points, this gone a give a lot of triangles :?
IdeasVacuum wrote:Mythos, the number of mesh faces does matter, it matters a lot, because viewing performance can be severely hampered. This is a known issue in CAD/CAM apps for example.

Marc, 1mm/px is not too bad? The main issue is the 3.1mm distance between slices.
I think it is possible to vary this on the machine itself, but 3.1mm is probably already the minimum distance. The more slices, the longer it takes and that has two issues - firstly, being scanned is stressful for the patient and secondly there are not enough machines to meet demand so the fastest possible scan session is always a requirement.
It's all money / time, so they slices the scan every 3.1 mm
What I did now, I pushed all the scans together of a 1 pixel distance,
this gives a deformed body, (3 times shorter) but for testing it works.

Re: CT/MR scans > 3D viewer

Posted: Tue May 06, 2014 9:41 pm
by Mythros
Well ok, yes. In THAT way, it DOES matter. HOWEVER. I think if we mess with the below example, we should be able to just remove SOME of the non-needed points, and connect the points as triangles, that DO matter.

For instance, I have messed with a demo in order to **TRY** and create depth, and I actually succeeded 75%.

I would possibly like to see if there is a way to create the effect of taking a cloth, and putting it on an ordinary, every day object such as an apple. The cloth would then assume the shape of the apple, therefore, creating a 3d model out of the "2d" image. Let me know when you guys come up with something.

Without further adieu, here's the source:

Code: Select all

Declare DrawMatrix()

#Mode = 7 ; #Mode = 1 or #Mode = 7

    Structure PointVertex
      x.f
      y.f
      z.f
      CompilerIf #Mode = 7
        Size.f ;--- Only if Mode = 7
      CompilerEndIf
      Color.l;d3dcolor
    EndStructure

ExamineDesktops()
MP_Graphics3D (DesktopWidth(0),DesktopHeight(0),0,3) ; Erstelle ein WindowsFenster #Window = 0
MP_Viewport(0,0,DesktopWidth(0),DesktopHeight(0)-5)

SetWindowTitle(0, "Pic_to_Mesh .. press Up, Down keys to move mesh forward/back | Left, Right keys to turn mesh left/right")

camera=MP_CreateCamera()

light=MP_CreateLight(2)
MP_LightSetColor (light, RGB(255,255,255))

texture2 = MP_CreateTextureColor(256, 256, MP_ARGB(0,255,255,255))
texture = MP_CreateTextureColor(256, 256, MP_ARGB(0,255,0,0))

plane = MP_CreatePlane(10, 10)
MP_EntitySetTexture(plane, texture2)
MP_RotateEntity(plane, 90, 0, 90)
MP_PositionEntity(plane, 0, -5, 0)

;MP_PositionCamera(camera, 0, 5, 20)
;MP_CameraLookAt(camera,0,0,0)

MP_PositionEntity(light, 0, 10, 20)
MP_EntityLookAt(light,0,0,0)
;==============================================================

UseJPEGImageDecoder()

; load image
Global image_id.i

image_id = LoadImage(#PB_Any, GetCurrentDirectory()+"woman.bmp")

;Dim image_pixel_data.b( ImageWidth(image_id), ImageHeight(image_id) )
Global size = Int(ImageWidth(image_id)*ImageHeight(image_id))
Global Entity= MP_CreatePrimitives (size, #Mode)

Global pointcloud_point_size.f = 1.50

DrawMatrix()
;==============================================================

MP_PositionEntity(light, 0 , 10, 10)
MP_EntityLookAt(light,0,0,0)

z.f=10
While Not MP_KeyDown(#PB_Key_Escape) And Not WindowEvent() = #PB_Event_CloseWindow
 
  If MP_KeyDown(#PB_Key_Up)
    z.f - 0.1
  ElseIf MP_KeyDown(#PB_Key_Down)
    z.f + 0.1
  EndIf
  
  If MP_KeyDown(#PB_Key_Left)
    MP_TurnEntity(Entity,0, 1, 0)
  ElseIf MP_KeyDown(#PB_Key_Right)
    MP_TurnEntity(Entity,0, -1, 0)
  EndIf
 
    MP_PositionEntity(Entity, 0, 10, z)
   
  MP_RenderWorld()
   
  MP_Flip ()

Wend


 Procedure DrawMatrix()
 i=-1
*Memory = MP_GetMemPrimitives(Entity)

StartDrawing( ImageOutput( image_id ) )

For y = 0 To ImageHeight(image_id) - 1
  For x = 0 To ImageWidth(image_id) - 1
    r = Red(Point( x, y ))
    g = Green(Point( x, y ))
    b = Blue(Point( x, y ))
   
    ;r = MP_Red(Point( x, y ))
    ;g = MP_Green(Point( x, y ))
    ;b = MP_Blue(Point( x, y ))
   
    i+1
   
    *myvertex.PointVertex = *Memory + i * SizeOf(PointVertex)
   
     *myvertex\x = x - ImageWidth(image_id)/2
     *myvertex\y = y
     
     If Y > ImageHeight(image_id)/2
        *myvertex\z = (MP_Red(Point( x, y )-8))
      EndIf
      
     *myvertex\Size = pointcloud_point_size
     *myvertex\color = MP_ARGB(0,r,g,b)
  Next x
  Next y
  StopDrawing()
  MP_ScaleEntity(Entity, 0.03, 0.03, 0.03)
  ;MP_MeshSetAlpha(Entity,2)
  MP_PositionEntity(Entity, 0, 0, 0)
  MP_RotateEntity(Entity,180,0,0)
 
  MP_CloseMemPrimitives(Entity)
  
  EndProcedure
I have also included the image I used:

Image


Rename the above image to "woman.bmp" and make SURE that the file type is INDEED bitmap & not JPG.

Also, can someone tweak this code so that it gives depth to more than just half the image?

Thanks again, and I hope to continue this research with you guys!

Sincerely,

Mythros