Elevation data for a new project

Home Forums GDS Support Elevation data for a new project

This topic contains 16 replies, has 5 voices, and was last updated by  Karen 11 months ago.

Viewing 15 posts - 1 through 15 (of 17 total)
  • Author
    Posts
  • #695

    Aakkerman
    Member

    I’m having trouble setting up a new model using .shp file with xyz elevation data. The points look great in GIS (have the right elevation values) but when I load them into the GDS some “pieces” of the original data come in with a single elevation value set for the whole region. The file is for a large area and has over 3,000,000 points and takes quite a while to load. When I got back to the file in GIS, the areas that come in with one elevation have their elevation value changed. Anyone else had this problem? Any ideas?

    Also, anyone know how to develop a .pts file in GIS – no one here has heard for such a file format. I thought maybe bringing in separate chunks of the elevation data might help the problem but I can’t seem to do it with shape files.

    #988

    Karen
    Keymaster

    You can use the program QGIS or ArcGIS to save your point data as an ASCII XYZ text file. You can get information from the website help for each of those programs to show you how to do it. QGIS is a little easier. I also have a contact I can give you who will do this work for a small fee. Email me directly if you’d like to use this service.

    #1104

    Hydrad
    Member

    I have spent the last few weeks trying to find a way to get elevation data into Flo-2d. I have ArcGIS Desktop 10.3 and I cant find any way of creating an ASCII point file from a raster. I tried QGIS and was able to convert the raster file using the Raster->Convert->Tranlate to create an ARCInfo ASCII Grid file. But Flo-2D will not read the file. After working for 30 to 40 minutes, it returns an error “Error reading file Elev.asc in line 0” Then the error -> “Error 14 Out of String Space”. Sometimes the second error I get is “Error # 62: Input past end of file.

    We are at our wits end. Any help would be appreciated.

    #1105

    Karen
    Keymaster

    Hi Hydrad,

    Would you like to try my new Plugin for QGIS? You can use your raster directly to get your grid element elevations using a point sample. Send me a message and I will give you the installer instructions.

    I will just assume you don’t have spatial analyst or 3-D analyst. Those are the tools that you use to convert elevation data using ArcMap. You can also do this in ArcGIS using a Point Sample procedure. If you create a grid system, you can import the topo.dat file to your map. Use a point sample to read elevation data from the raster at a specific point.

    I have some training videos that might offer some insight to this process. It isn’t difficult but I can’t remember right off hand how to do it in ArcMap. This training series will show you how to manipulate elevation data in 3 different GIS programs.

    https://flo-2d.sharefile.com/d-saa9c5e6fd7d44a0b

    #1106

    Hydrad
    Member

    Hi Karen, we have Spatial Analyst and 3D Analyst, but I have ArcGIS not ArcMap. ArcMap is the more expensive version and includes more functions than ArcGIS. Seven thousand was all we could justify putting into a GIS. I tried using 3D Analyst to create a shape file of points. I used Conversion->From Raster->Raster to Multipoint, but the multipoint would never pickup the elevation data from the raster. This was the only function in 3D Analyst that seemed reasonable to convert the raster to. The other options were Raster to Raster Domain, which only had line and polygon class, or Raster to TIN, but I dont think your program reads TIN’s. I didn’t see anything in Spatial Analyst that converted to ASCII Grids or point shape files.

    I would like to try your plugin for QGIS, although I don’t know what you mean by point sample. What is the file format of the topo.dat file that you are referring to?

    You can send the plugin to office@hydrologic-systems.com.

    Tom

    #1107

    Hydrad
    Member

    Hi Karen, I went through your video on preparing elevation data sets. I have prepared an ASCII file and a point XYZ file that are formatted just like you have in your powerpoint. I have opened them up in textpad and looked at the format to make sure they follow the standard formatting. But I still get the same errors from GDS when I try to enter either file.

    Below is a snippet of the xyz file:

    5974206.6847428735 2207666.0723353699 95.038932800292969

    5974211.6847428735 2207666.0723353699 94.979225158691406

    5974216.6847428735 2207666.0723353699 94.904998779296875

    5974221.6847428735 2207666.0723353699 94.932106018066406

    5974226.6847428735 2207666.0723353699 95.290351867675781

    Below is the header of the ASCII Text File, the data row is too long to include:

    ncols 2642

    nrows 1636

    xllcorner 5974204.184742873500

    yllcorner 2199488.572335369900

    cellsize 5.000000000000

    NODATA_value -3.4028230607370965e+038

    …. The data follows from here ……..

    It seems that the files follow the correct format, so I am not sure why the program is crashing.

    #1108

    Karen
    Keymaster

    Have you considered that your data is too big for the GDS? It is a 32-Bit program and it will run out of memory if you try to load too many points or build to many rows and collumns in your grid system.

    Start with a smaller sample of data. If you think you are getting a memory error, use the Interpolate from Multiple Files option. It is the option we use for LiDAR data and there is not limitation.

    Also, use a simple polygon shapefile to start your GDS. So instead of starting from a point shapefile, you are starting from a polygon of the Area of Interest. Then try and build a grid to make sure you aren’t using too many grid elements. For example, if you use a 5 m grid element size, you might also run out of memory.

    #1109

    Hydrad
    Member

    I have gradually brought it down to a 20 ft spacing for my elevation data, which gives me 265,000 elevation points. I still get the same problem.

    I create the project from a polygon shape file

    Then I bring in my aerial photograph

    Then I bring in the elevation data, and that’s where it stops. No matter how coarse and small my elevation grid, the program spends hours working on it and then gives me the error.

    #1110

    Karen
    Keymaster

    OK but you do not need to bring the data into GDS. That is the point of the Multi-File Interpolation option. You can choose the second interpolation method and call an xyz elevation text file with an extension of *.txt. Please see the GDS manual for help and review the training video that I sent to you. Don’t use a shapefile. Extract the points from your shapefile into a text file. You can write x and y coordinate to the attribute table and save the table to a csv file. Then rename the extension from .csv to .txt. Remove any header data and make sure only x,y,z data is in the file. The data can be comma or space delimited.

    #1111

    Hydrad
    Member

    I tried to use the “Interpolate From Multiple Elevation Files” command. I assume the file that it reads in (elevfiles.dat), lists the file name(s) that you want to interpolate from, so I put in the 20-ft grid file name, an x-y-z file that contains the 260K points (Miller20ft.pts). The program churned for about an hour then returned the following error:

    Error 1906101111: Error while interpolating LIDAR files

    Then:

    Error # 0:

    I have no idea how to proceed from here. It seems like we have tried just about everything.

    #1112

    Hydrad
    Member

    All of the elevation datasets that I have tried to use, were created with QGIS, and look fine in that program, as well as in ArcGIS, so I am not sure what the problem is.

    #1113

    Hydrad
    Member

    After 3 weeks, I think that I finally figured it out. Despite what the Forum postings say, the output files from QGIS are NOT compatible with Flow 2D. But, if you bring those files into Excel, and save them as a space delimited text file then Flow 2D will read them. That little tidbit cost us about 10k in labor hours. You should correct the forum postings.

    #1114

    Karen
    Keymaster

    If you review the training videos you will see that I show the difference between ASCII text and UTF8 text. QGIS exports UTF8 text and that has a different carriage return than ASCII text. I converted it using a text editor. I set the view to hexadecimal and did a search for the carriage return characters. Then I did a find and replace and replaced all of them with the ASCII carriage return. There is also a converter built into windows computers that you can access via the command prompt. I might have it in the video but I can’t remember. You can search for it online. It is basically a matter of calling the file, processing the conversion into a new file.

    If you don’t want to mess with any of that, I recommend MapMart. You can purchase your data for a small fee of like $75 US dollars and it will come in the exact format you need.

    #1117

    Hydrad
    Member

    I was trying to make use of the LIDAR data that we have for the site. I processed it with ArcGIS, but just couldn’t get any output that Flow2D could read. But I think that by processing the data in ArcGIS then using QGIS to read the output of ArcGis and then getting Excel to read the output of QGIS and output it to a standard text file will work.

    #1411

    ekarch
    Member

    I successfully used ArcMap to create an x,y,z ASCII from LiDAR data and imported into a new Flo2D project. I found this website to be useful. The article title is “Raster to XYZ in ArcMap” written by CHAD on Aug 15, 2011 and was found on the website dv-proj.com.

    Raster to XYZ in ArcMap

    I followed the author’s process to the end in step 6 where I opened the .dbf file in Excel and saved it to a .csv file. In doing so, I used the Excel dropdown filetype menu to select MSDOS csv instead of CSV UTF-8. After that, I opened windows explorer and renamed the file extension as .pts. Then in Flo2D, I used File > New project > from DTM points.

Viewing 15 posts - 1 through 15 (of 17 total)

You must be logged in to reply to this topic.