NEOS Server FAQ

The NEOS (Network-Enabled Optimization System) Server is a free internet-based service for solving numerical optimization problems. Hosted by the Wisconsin Institutes of Discovery at the University of Wisconsin in Madison, the NEOS Server provides access to more than 60 state-of-the-art solvers in more than a dozen optimization categories. The NEOS Server offers a variety of interfaces for accessing the solvers, and jobs run on distributed high-performance machines enabled by the HTCondor software. The NEOS Server is available free of charge for everyone, anywhere in the world.

There is no specified limit to the number of jobs that a user can submit. However, we ask all users to submit no more than 10-15 jobs at a time. Users should wait for these jobs to finish before submitting another batch. NEOS is a free and volunteer service, and we don’t have the capacity to handle large-scale submissions, which may cause significant problems due to network timeouts.

Solvers available on the NEOS Server are not available for download from this site. Some of the solvers are commercial products, which are made available for use for free through the NEOS Server. Others are NEOS Server implementations of academic codes or open source codes, which may or may not be available for download. For information about a particular solver and its availability, check the solver’s interface page on the NEOS Server site. Also, you can do a web search on the name of the solver or the name(s) of the developer(s).

Most of the solvers are hosted at the University of Wisconsin in Madison but some are hosted at other institutions. For the solvers that are hosted at the University of Wisconsin in Madison, the hardware specs are listed below. Please note that jobs do not run on prod-web-1 or prod-sub-1.

• prod-exec-1 is a Dell PowerEdge R430 server with the following configuration:
  • CPU – 2x Intel Xeon E5-2698 @ 2.3GHz (32 cores total), HT Enabled
  • Memory – 192GB RAM
  • Disk – 4x 300G SAS HDDs setup in RAID5
  • Network – 1Gb/s Ethernet
• prod-exec-2, prod-exec-3, and prod-exec-4 are Dell PowerEdge R410 servers with the following configuration:
  • CPU – 2x Intel Xeon X5660 @ 2.8GHz (12 cores total), HT Enabled
  • Memory – 64GB RAM
  • Disk – 2x 500GB/2TB SATA HDDs setup in RAID1
  • Network – 1Gb/s Ethernet
• prod-exec-5 is a Dell PowerEdge R420 server with the following configuration:
  • CPU – 2x Intel Xeon E5-2430 @ 2.2GHz (12 cores total), HT Enabled
  • Memory – 64GB RAM
  • Disk – 2x 1TB SATA HDDs setup in RAID1
  • Network – 1Gb/s Ethernet
• prod-exec-6 is a Dell EMC PowerEdge R440 server with the following configuration:
  • CPU – 2x Intel Xeon Gold 6140 @ 2.3GHz (36 cores total), HT Enabled
  • Memory – 512GB RAM
  • Disk – 2x 1.5TB SAS SSDs setup in RAID1
  • Network – 1Gb/s Ethernet
• prod-exec-7 is a Dell EMC PowerEdge R440 server with the following configuration:
  • CPU – 2x Intel Xeon Gold 5218 @ 2.3GHz (32 cores total), HT Enabled
  • Memory – 384GB RAM
  • Disk – 4x 500GB SATA SSDs setup in RAID5
  • Network – 1Gb/s Ethernet

• Czyzyk, J., Mesnier, M. P., and Moré, J. J. 1998. The NEOS Server. IEEE Journal on Computational Science and Engineering 5(3), 68-75. This paper discusses the design and implementation of the NEOS Server.
• Dolan, E. D. 2001. The NEOS Server 4.0 Administrative Guide. Technical Memorandum ANL/MCS-TM-250, Mathematics and Computer Science Division, Argonne National Laboratory. This technical report discusses the implementation of the server and its use in detail.
• Dolan, E. D., Fourer, R., Moré, J. J., and Munson, T. S. 2002. The NEOS Server for Optimization: Version 4 and Beyond. Technical Memorandum ANL/MCS-P947-0202, Mathematics and Computer Science Division, Argonne National Laboratory. This preprint, available for download in PDF format, describes the advances in version 4 of the server.
• Dolan, E. D., Fourer, R., Goux, J.-P., Munson, T. S. and Sarich, J. 2008. Kestrel: An Interface from Optimization Modeling Systems to the NEOS Server. INFORMS Journal on Computing 20(4), 525-538. This paper gives details of the design and use of the Kestrel interface to the NEOS server. It is available for download in PDF format.
• Dolan, E. D. and Munson, T. S. 2001. The Kestrel interface to the NEOS Server. Technical Memorandum ANL/MCS-TM-248, Mathematics and Computer Science Division, Argonne National Laboratory. This user’s guide to the Kestrel interface is available for download in compressed postscript format.
• Ferris, M. C., Mesnier, M. P. and Moré, J. J. 2000. NEOS and Condor: Solving Nonlinear Optimization Problems over the Internet. ACM Transactions on Mathematical Software 26, 1-18. This paper discusses the use of Condor, a distributed resource management system, as a provider of computational resources for NEOS. The paper is available for download in PDF format.
• Gropp, W. and Moré, J. J. 1997. Optimization Environments and the NEOS Server. Approximation Theory and Optimization, M. D. Buhmann and A. Iserles, eds., Cambridge University Press, pages 167-182. This paper discusses the NEOS Server as a problem-solving environment that simplifies the formulation of optimization problems and the access to computational resources.

Cite the three papers listed here.

We are always interested in learning about new solvers that would extend the capabilities of the NEOS Server. To submit your solver for evaluation and consideration, contact us by email at support@neos-server.org.

Every job submitted to the NEOS Server is limited to 3 GB of RAM and a maximum run time of 8 hours. Any output produced by the solver is also limited to 100MB. Results larger than this will be truncated. In addition, multithreaded solvers are limited to a maximum of 4 threads per job.

The User’s Guide to the NEOS Server provides detailed instructions on how to submit jobs to the NEOS Server but here is a brief overview of the steps.

1. Select an appropriate solver for your optimization problem. The available solvers are listed by category. If you have already have a written formulation of your model, your choice of solver may be somewhat limited by the input formats available for the solvers.
2. For the solver that you have selected, choose an appropriate input format (e.g., modeling language, programming language, solver-specific input format) and prepare the model, data and additional (e.g., commands) files as specified.
3. Select a method of submission for your job: web interface, XML-RPC, or Kestrel. If you use the web interface, you can upload the model, data, and additional files as text files. If you plan to use XML-RPC or Kestrel, consult the documentation on the NEOS Guide site.
4. Submit your job to the NEOS Server. When the NEOS Server receives the submission, it assigns a job number and a password and it queues the job for scheduling on one of the machines. When the job is received by a machine, the submission is unpacked, the driver processes the submission, and the optimization code executes.

Recently, the NEOS server has had several incidents involving users who submit a very large number of jobs in a short period of time. This has caused issues for the NEOS team as well as for other users utilizing NEOS. Requiring an email with the job submissions would allow us to contact the users who may be inadvertently causing issues on the server.

Yes. NEOS runs an XML-RPC server to communicate with clients for submitting jobs and retrieving results. As of July 2016, the NEOS XML-RPC server will accept HTTPS submissions at the URL https://neos-server.org:3333. Please update your client programs with the new URL to avoid problems with submissions. HTTP submissions to the URL http://neos-server.org:3332 will be phased out. There are client libraries written in Java and Python; a client library in Javascript is under development. See the NEOS API page for more information.

Yes! SolverStudio is an add-in for Excel 2007 and later on Windows that allows you to build and solve optimization models in Excel using many optimization modeling languages. SolverStudio allows models built using AMPL and GAMS to be solved using the NEOS server. For more information, see StudioSolver & NEOS.

The NEOS Server cannot accept input files larger than 16 MB. The 16 MB limit applies to submissions via the web interface, email and XML-RPC.

You cannot submit a compressed file of your entire job submission but you can compress and submit the individual files (e.g., model, data, commands). The NEOS Server checks each file to see if it is compressed, so you can compress any data for which the interface has a Browse option. The NEOS Server supports .gz and .zip.

For security reasons, jobs are not allowed to read additional input files after they have been received by NEOS, and GAMS jobs cannot do a $call or execute statement.

In general, jobs are not allowed to write additional output files; one exception is that GAMS jobs are allowed to write a text file, specifically named results.txt. This output file can be retrieved from the NEOS Server in the html output directory for the job. See the FAQ about retrieving GDX output for details on locating the html directory.

There are several other methods that can achieve a similar effect to writing an output file.

If you have your own copy of AMPL or GAMS, you can have the modeling language software read your files locally before submitting your job to NEOS via the Kestrel interface. When the solver is finished, the results are returned to your local AMPL or GAMS environment where you can write whatever file(s) you need. More information about Kestrel is available on the NEOS Server site.

If you are using the NEOS XML-RPC interface to submit your job to NEOS, you can write a simple program to retrieve the NEOS output and write it to a file. More information about writing client code for the XML-RPC interface is available in this section.

It depends on the solver. Some solvers require all of the subroutines to be in a single file, while others allow separate files. Consult the solver’s interface page for specific details.

From the NEOS Server page, click on the View Job Queue and Job Results link to go to the job admin page. Then, select the View Job Queue button and click Submit.

From the NEOS Server page, click on the View Job Queue and Job Results link to go to the job admin page. Then, enter the job number and the job password, select the Kill or Dequeue Job button and click Submit.

By default, jobs submitted to the NEOS Server are assigned as long jobs, which means that they can run at most 8 hours. There are two important things to know about long jobs. First, results for long jobs do not stream. For information about how to retrieve results, see the answer to the question I submitted a job to NEOS but I never received the results. How can I retrieve the results? Second, partial results may be returned for a timed-out job but this is ultimately dependent on which solver / input format was utilized. In some cases, the administrator of a solver will set a shorter maximum time; in other cases, an administrator of a solver will set a maximum number of iterations (the job will terminate at the iteration limit even if it is less than 8 hours). If you expect that your job will approach or exceed the 8 hour limit (or the iteration limit), you should use the solver’s options to set a maximum time value less than 8 hours. That way, when the job terminates, you will receive a log of the progress made.

Jobs submitted to the NEOS Server can be assigned as short jobs, which means that they can run at most 5 minutes. Use the <priority> tag to assign a priority to a job. Results for short jobs do stream.

GAMS Users: GAMS sets a default limit of 1000 seconds but this limit can be extended by adding a “reslim” statement before the solve statement. For example, to increase the limit to 2000 seconds for a model with the name “my_model”, you would include the statement my_model.reslim = 2000;

Jobs submitted to the NEOS Server are limited to 3 GB of RAM. Partial results may be returned for jobs that were terminated due to exceeding the allowable memory limit however the user should not assume this will be universally true for all solvers across all input formats. If your job terminates without returning any results, most likely it required too much memory.

It is often with mixed-integer programming (MIP) problems that you would have issues with memory. MIP solvers accumulate and store in memory information about the branch-and-cut tree as they progress. At some point, the amount of memory required for the tree information may exceed the amount of memory available. Most MIP solvers offer a node file storage feature to store tree information in files in order to reduce the amount of memory needed.

For Gurobi, you can use the parameter nodefilestart to control the node file storage feature (NEOS limits jobs to 3GB). We also recommend nodefile=3 to compress the
information saved to disk. For more information, see the documentation on the Gurobi website.

For CPLEX on the GAMS platform, you can set the following parameters:

set workmem 128
set mip strategy file 2
set mip limits treememory 10000

This limits memory for nodefile to 128MB; if it starts getting bigger, it writes the nodes to a file. The last line sets the maximum size of the node file to 10GB.

If your job completed, you can view the results from the job admin page. From the NEOS Server page, click on the View Job Queue and Job Results link to go to the job admin page. Then, enter the job number and the job password, select the View Job Results button and click Submit. Alternately, registered users can access their most recent jobs from the MyJobs page as long as the job was submitted while the user is logged in.

If you submit large and difficult problems, often you will receive results that only indicate the job was scheduled and started running, for example:

    MODEL.gms -limcol=0 -miqcp=cplex -suppress=0 -al=0 -ao=0 -limrow=0
    -threads=4 -rmiqcp=cplex -lf=gams.logfile -ps=0 -qcp=cplex -pw=79 -lo=2
    -execmode=2 -o=solve.out -cerr=10 -lp=cplex -mip=cplex -rmip=cplex
    -relpath=1
    Calling GAMS.

    File exists
    Executing on neos-7.neos-server.org

The most common reasons that you would receive this result and no other result are that your job exceeded the time limit (8 hours) or the memory limit (3 GB). Partial results may be returned for jobs that are terminated due to the time or memory limit but this is ultimately dependent on which solver you are using. See the question How long can my problem run using NEOS? or the question What is the maximum amount of memory available for a job submitted to the NEOS Server?. We are working to provide more meaningful error messages in the future.

The amount of output that streams for a short priority job depends in part on the input method and the solver. For GAMS jobs, you must check the box “Return log file” for the solver to stream information. For AMPL jobs, you must specify output options for the solver using a commands file. Here are suggestions for some MILP solvers:

• CPLEX: mipdisplay=1
• Gurobi: outlev = 1 (default setting)
• MOSEK: outlev = 2
• XPRESS-MP: outlev=1

You need to specifically request that the solver return the values of the decision variables. The format of the request differs by solver and by input, so you will need to refer to the solver’s documentation for specifics. For AMPL, see the answer to the question How do I get AMPL to return the values of the decision variables?

If you have not received any response from the server within a reasonable amount of time, you probably will not. Upon receipt of your job request, the NEOS Server should send you an immediate message with the job number of that request. Failure is often attributed to one of these causes:

• The server will only respond to messages sent To: neos@neos-server.org. Command line mailers should only be used with the option to include recipients on a To: line in the message.
• The message must be type text/plain, the simplest form of email message.
• Your mail program may be pre- or post-appending information to your message. To avoid information appended at the end, add the END-SERVER-INPUT tag to your data.

If the NEOS Server cannot parse the first line correctly, you should receive mail from the server with the offending line and the entire message below. If the NEOS Server is not responding at all to your email submissions, submit a test problem (choose one of the sample submissions for the solver) to see if the server can understand your mail program.

There are a number of resources available on the AMPL website. There is an AMPL FAQ as well as information and documentation for the solvers that work with AMPL.

The NEOS Server renames the model and data files that are submitted, so you will get an error if you try to reference the name of the model file or the data file in an AMPL command. NEOS automatically loads the model and data files, so it is not necessary to write include statements in your AMPL command file.

If you have include or reset commands in your AMPL commands, you are likely to get an error message:
Executing AMPL.
Error:
processing data.
Error: amplin, line XXXX (offset XXX):
processing commands.
Error: XXXX is not defined
No variables declared.

This error arises because NEOS does some preprocessing on the submitted model, data and command file. It concatenates everything into a single file and adds some options, such as specifying the solver, before sending it to be scheduled. Certain commands, particularly include and reset cause problems during the preprocessing. To resolve the error, remove the command(s).

This error indicates that your PC cannot connect to the NEOS Server, perhaps because of a company firewall or the policies on your PC blocking external access. You should talk to your network administrator; he/she may be interested in the Internet Access notes on the SolverStudio Download & Install page.

In general, it means that Xpress has encountered an error in the solution process. However, we have found that the main cause is usually that Xpress cannot find a feasible point. The error may look something like this:
XPRESS-MP 12.13: maxtime=7000
Error preparing solution file
? 85 Error: File not found: _amplpb.sol
?122 Error: Corrupt solution file.
exit code 1

If you are using a solver that accepts command files, submit a file that contains a solve command followed by the commands to display results. For example, to create a table of all variables with their lower bounds, optimal values, and upper bounds, use the following commands:
solve;
display _varname, _var.lb, _var, _var.ub;

You can retrieve the output from the NEOS Server by going to the html output directory for your job. Jobs are grouped into directories by 10,000s. To get the output directory, round down the job number to the nearest 10,000. Then, go to

https://neos-server.org/neos/jobs/directory_number/job_number.html

At the bottom of the page, you will find the link(s) to the additional output. If the link is missing or broken, you can use the following specific URL where job_number is the job number and job_password is the associated password:

https://neos-server.org/neos/jobs/directory_number/job_number-job_password-solver-output.zip

The problem may be that your output file is too large (too many lines). To suppress some of the output (e.g., not printing the constraint values), add commands to the top of the GAMS file. For example:
option limrow = 0;
option limcol = 0;
option solprint = off;
$offlisting $offsymxref offsymlist

If you are using MINOS-GAMS via NEOS, MINOS 5.51 is selected by default, and NEOS names the option file that you submit as minos.opt. Specifying option nlp=minos5; in your submission has the effect of requiring the option file to be named minos5.opt. The conflicting names, minos.opt versus minos5.opt, cause the error.
*** Error Cannot open parameter file "/scratch/neos/solver/jobs/XXXXXXX/minos5.opt"
*** Error Error code = 2; No such file or directory
Remove the command option nlp=minos5 to resolve the problem.

This requires “pre-processing” locally on the user’s part. NEOS does not accept xls uploads. Below is a brief example that will read in a xls file (say gdxexample.xls) and output a gdx file (“in.gdx”).

$onecho > extract.txt
set=a rng="Sheet1!A2" dim=1 rdim=1 values=NoData
set=b rng="Sheet1!B2" dim=1 rdim=1 values=NoData
set=reg rng="Sheet1!C1" dim=1 cdim=1 values=NoData
set=foo rng="Sheet1!A8" dim=3 rdim=3
par=c rng="Sheet1!A1" rdim=2 cdim=1
output=in.gdx
$offecho

$call 'gdxxrw gdxexample.xls @extract.txt'

The resulting gdx file can then be uploaded to NEOS and the GAMS model should include code to load the xls data from the gdx file, for example,

$gdxin in.gdx
$load a
$load b
$load nreg=reg
$load data=c
$gdxin

More information can be found on the GAMS documentation.

If you receive a blank screen after submitting a job, it almost always means that there are non-ASCII characters in your job file(s) or that there is a syntax error in the input file. The blank screen results because the NEOS Server cannot properly parse the XML file. Check your input for accented characters, Greek letters or mathematical symbols and delete or replace them. If you are having trouble finding where the characters are in the file, try saving it as a text file or opening and saving it in a plain-text editor.

By default, the priority of a job is long; output for long jobs does not stream and jobs can run for at most 8 hours. When the job has completed, the results will be emailed to you, so always provide a valid email address. If you do not receive an email, you can go to the job admin page to view the queue or job results. Also, see the answers to the questions I submitted a job to NEOS but I never received the results. How can I retrieve the results? and How long can my problem run using NEOS?

Jobs submitted with the default priority of long can run for at most 8 hours. Jobs submitted with the priority short can run for at most 5 minutes. See the answer to the question How long can my problem run using NEOS? for more information.

These errors mean that your job submission has reached/exceeded the maximum of 3GB of RAM.

This error may be an indication that your job submission contained non-ASCII symbols from a word processing program or other program. Try saving the file as a text file to get rid of the formatting symbols.

The time out error arises when NEOS receives too many connection requests within a short period of time. There are two options for your code: (1) submit a request for the final results and wait for them to return or (2) loop over the status of the job but place a wait in the loop so that it doesn’t loop more than once every 5 seconds.

This error may occur if you are using a third-party application to access NEOS. The issue has been resolved in OpenSolver 2.8.0, so please upgrade if you are using an older version.

A run time error indicates that an error occurred during the solution process, often a syntax error or a floating point exception (overflow or subscript out of bounds). In most cases, the solver should detect the error and report a message about the problem. In rare cases, the solver may terminate unexpectedly with no error message; please contact us to let us know about the problem.

The NEOS Server undergoes periodic maintenance and is upgraded from time to time, so the page may have moved. Also, solver administrators may choose to disable a solver at any time. If the solver appears on the list of solvers but you cannot access it, please contact us.

There is a bug in Kestrel/GAMS 30.x, which has been fixed in GAMS 31.0. Please refer to this post on the GAMS forum for a client-side fix.