Problem Synthesis Draft 7

Problem Synthesis Draft 4

Problem Synthesis Draft 3

Problem Synthesis Draft 2

Problem Synthesis: Troubleshooting InVEST Toolbox

Shawn Dayson 2010 Nov 20

Using modified Hargreave's to estimate ETo.
I spent some time trying to estimate ETo using the modified Hargreave's equation as given in the user manual. My thinking was that this would be interesting because of microclimate situations, and the resolution would be much finer than the ETo from the UN. The user manual gives no guidance how to extrapolate the Modified Hargreave's equation to an average annual ETo. As the equation is now you would get only the ETo on a single day, because the RA variable is for a single day. To extrapolate this out over a year you will need some sophisticated averaging.

Nutrient Export coefficients sample data is acceptable but not comprehensive. (Chris)
Lin provides a great lit review of the possibilities out there: Dodd & McMahon (1992) seems preferable than Reckhow et al (1980) to me.

It appears most of the sample data sections could be more comprehensive of what literature is out there. I realize the more people who use InVEST, the longer this list will end up being. However, it would give people a better idea of what is out there.

Pollination manual instructions inconsistent with sample data.
Manual Says:
For the N_cavity/ N_ground values, user chooses one LULC to have a value of 1 (meaning the best place for this habitat) and the rest of the LULC's are on a relative scale between 0-1. The sample data is not scaled this way and I wonder if it changes the output.

Hydrology models require more than a Beginner's/Intermediate skill set.
I (Shawn) spent many hours trying to figure out how to do one thing in ArcMap or Python. I do not think this is in the range of expertise that is suggested by the manual.

Model builder can be used for running multiple data sets in the Hydrology Models:

I (Shawn) discovered that I could run multiple inputs at a time by creating a model in Model Builder. This is an effective way of testing the model to see what results may be yielded.

Watersheds raster will not mosaic while running the Nutrient Retention model:

Error combining sub-watershed outputs: Failed to execute. Parameters are not valid.
ERROR 000732: Input Rasters: Dataset E:\InVEST\UNRun3\Intermediate\
export_sub_wsheds_A\w_exp_3_A does not exist or is not supported
Failed to execute (Mosaic).

Error running script.
<type 'exceptions.Exception'>:
Failed to execute (Nutrient_Retention).

Solution: Convert watershed raster to a polygon file, dissolve all fields, convert back to raster.

If a DEM is used once and an error occurs, the entire Hydro_layers folder must subsequently be deleted.
  • Error: A non-cardinal flow direction value has been encountered in E:....Hydro_layers\Flow_dir

Just by looking at the DEM you're unlikely to see anything obviously wrong with it. Often, just doing a Fill on the DEM gets rid of this problem. Try doing a Fill then manually do Flow Direction and see if it helps. Then, if you save the filled DEM in the same folder as the original DEM, delete the Hydro_layers folder so the model will re-generate a new Flow_dir using the filled DEM that you specify in the input window. (Otherwise, the model will see that the layers already exist and will not generate new versions with your new DEM.)

None of the DEM suggestions in the model manual resolve DEM problem.

The manual suggests running Sink tool and then Fill tool from ArcToolbox. This does not work as running fill leads to a single solid layer. I was able to make a successful change by only running Fill.

Biophysical tables cannot be uploaded.

Note from George: This complaint is repeated in several places - do not require DBF files - they are difficult to work with - XLS is much easier - you should make things easy for the user, not the programmer.

The only way I can create a DBase file is by directly editing the attribute table from the InVEST sample data. I have tried to create excel files, DBase files using Access (DB III, DB IV, and DB V), and using ArcCatalog. This is not a problem with other models where an excel file can be used.

Solution: Export sample data DBase file selecting only the first 7 rows (we only have 7 LULC types.)

Pollination model failed : DLL complaint

I am having trouble just opening the sample data model. It fails and I get a message: 'exceptions.ImportError'. the DLL load failed and the specified module could not be found. I installed the additional requirements from the manual, so I am really confused.

Yes, I (george) did get pollination running on the computer in our classroom (the one in front). The troubles I had and the solutions are detailed here ...
You can also try ... Go to the invest toolbox (where you downloaded the model to begin with), log in, and poke around in the user forums - that's how I fixed my problem with the pollination model. User forums are the Discuss part of the navigation bar. If there's nothing already out there, post a new message to the forum and see if you can get a response.

example in step # 4 of the users manual - make sure there is NOT a space here between the semicolon and C:\....
This is how the example reads:
C:\Program Files\soft; C:\gdalwin32-1.6\bin
It should be:
C:\Program Files\soft;C:\gdalwin32-1.6\bin

Creating .dbf issues

Summary of complaint - working with dbfs is unpleasant - the model should take xls files everywhere (some modules do, some don't) - george

Hi Everyone! Meg here,
I just wanted to share a problem that I encountered while getting the carbon model running. (This may not apply to other models I'm not sure).
Anyways, the Carbon model requires carbon pools data input in .dbf form. I was working in windows 2007, which no longer has the option to save an excel file as .dbf I tried downloading a converter but this was more trouble than it was worth.
To solve this issue, I transfer the .xslx file to my other computer which has an older excel file and converted it there, then transferred it back to add to arcmap.
So if you are dealing with that issue I suggest that's the best workaround.
Thanks, meg

From George: For most other such files the manual says .dbf or .xls files will work - you might try a .xls file (save as .xls from Excel 2007). Kevin has tried this for pollination and it works. If it doesn't work for carbon, Meg's workaround is good.

Carbon Model Error message

Everyone, meg again, I got the carbon model all set up with some made-up numbers I created and the 2001 LULC map.
The first time I ran this model I got an error message:\
Error reclassing C_ABOVE Completed script Carbon... Failed to execute (Carbon_1). End Time: Mon Oct 04 11:31:48 2010 (Elapsed Time: 20.00 seconds).
I am going to play around with it more, just wanted to share in case someone else has encountered a similar issue and may know what the problem is?
Thanks, meg
PS: The whole list of info: Executing (Carbon_1): Carbon C:\EcoSrvcsInvest\Carbon C:\EcoSrvcsInvest\Base_Data\NLCDchangeUpperNeuse\unncld2001 2001 2000

Biodiversity won't run unless workspace is C:InVEST\Biodiversity

If I try to run the program with a workspace of D:\TestData, I get this message ...
Error creating habitat layer
... and the script terminates.

If I change it back to C:InVEST\Biodiversity and leave all else the same, the habitat layer is created (and then I have other problems because my threat layers are in D:TestData\Input)
george hess 2010 Oct 11

addendum 2010 Oct 27 - This might be a problem because I'm running on a virtual computer. It is possible that I am running out of disk space on a network drive. Suggest testing on a regular computer setup to confirm.

Arc attribute table columns hide digits

Be aware that if you open an attribute table in Arc (931) that it sizes the column based on the size of the column-heading text. If you have a number longer than the column is wide, the right-most digits are simply not seen unless you manually widen the column. You cannot tell if any digits are missing because the numbers do not use commas and there is no ... or ### indication that the field is too small. Strongly recommend that you manually widen the column if any of the numbers seem to fill the column.

george hess 2010 Oct 14

Why can't InVEST save previous dialog entries?

When running a model repeatedly, it would be nice if InVEST saved the previous entries in the dialog box rather than reverting to the sample dialog entries. For example, I had to run the biodiversity module 25 times for a sensitivity analysis. Each time I ran it, I had to refill every single dialog box, even though none changed from run to run (values in the threat and sensitivity files were being changed). Sure would have been nice to not have to do that.

george hess 2010 Oct 14

Biodiversity manual table could use further explanation

In the Version 1.005 manual, biodiversity section, p31-32, there is a table of degradation sources from Czech et al 2000. There is, however, no explanation of how it can be used. I believe one could create a relative threat level table for these threats (for the required threats input file) by dividing each threat by the maximum value of 305 (for interaction with non-natives). In other words, the threat for non-natives would become 1; for urban 275/305 = 0.9; for ag 224/305 = 0.73; and so on. Explaining this (assuming I have it right) would help users think about.

george hess 2010 Oct 18

Biodiversity - need for both Threat and Sensitivity unclear (and confusing)

The need for two values to modulate the level of habitat degradation from each threat is unclear and somewhat confusing. At the least, the difference should be explained clearly in the manual.

My interpretation is that the relative threat levels are an overall assessment of how damaging one threat is relative to others, and without regard to habitat type. It allows you to say that, for example, urban is twice as damaging as pasture, in general. The threat file is also where maximum penetration for the threat is defined, WITHOUT REGARD to habitat type (which seems a bit odd).

The sensitivity allows this to be fine-tuned for each habitat type. So, for example, you can say that forest is 3 times as sensitive to urban as it is to agricultural threats (despite what it says in the threats file).

I guess I'm not sure I see the point of this separation and wonder why this is needed. First, the whole point of the model seems to be for situations where data are scarce - and yet the model requires, essentially, two separate judgements for each threat; that seems contradictory to me. Second, allowing these two values creates (unneeded?) complexities and interactions when the model runs. Third, it is possible for a user to enter contradictory information about the threats - sensitivity values can be very different than the relative threat values, and it's not clear why that would be and how that plays out in the results. Finally, the two input files - threat and sensitivity - get really confusing with the file name requirements.

I wonder why this is not all handled in a simpler manner. Leave the threat file as it is; and rename the sensitivity file to habitat, discarding the threat columns in that file and keeping just LULC #, name, and habitat value.

george hess 2010 Oct 18

output_LULC pain

I just wanted to complain that the default lulc_output raster file is a pain. I was trying to run the model several times with the same session of arcmap and the lulc would replace itself, loosing my previous maps. It is also annoying that every time you open the invest toolbox it is populated with default stuff that I have to delete.

Note - This is essentially same complaint as Why can't InVEST save previous dialog entries?

Nutrient Model creates files outside of work directory

If the work directory is C:\InVEST\watertst why is the script creating hydro layers in D:UNData ? It should stay in the working directory. Problem for me is there's not enough space on D: to support the work files. Also, programs should never create files in a user's input directories - those should remain untouched.

Error message below - note attempt to create files outside of workspace.

george hess 2010 Oct 26

Executing: Nutrient_Retention C:\InVEST\watertst D:\UNData\UNDEM_SH_Meter1.img C:\InVEST\watertst\Service\wyield D:\TestData\Input\UNNLCD2001f.img D:\UNData\Watershed2.img D:\UNData\Watershed2.img D:\UNData\UN.gdb\UN_Biophysical_Table D:\UNData\UN.gdb\UN_Water_Purification true # 1000 MINOF # Start Time: Tue Oct 26 22:26:44 2010 Running script Nutrient_Retention...
Validating arguments...
Creating hydrology layers... Error processing hydrology layersERROR 999999: Error executing function. RasterDataset::SaveAs failed (ERROR: #1007) eimg_FileCopy failed (ERROR: #834) RFInstanceFileCopy failed (ERROR: #7632) FileCopy function failed (ERROR: #2450) ERROR 010240: Could not save raster dataset to D:\UNData\Hydro_layers\Flow_dir with output format GRID. Failed to execute (FlowDirection).
Error running script. : Failed to execute (Nutrient_Retention). End Time: Tue Oct 26 22:28:03 2010 (Elapsed Time: 1 minutes 19 seconds)