NEOS Server FAQ

Welcome to the newly updated NEOS Server FAQ! Listed below are common questions and answers about using the NEOS Server. Click on the section name to jump to the corresponding section.

NEOS Server and the solvers

  1. What is the NEOS Server and who may use it?
    The NEOS (Network-Enabled Optimization System) Server is a free internet-based service for solving numerical optimization problems. Hosted by the Wisconsin Institutes of Discovery at the University of Wisconsin in Madison, the NEOS Server provides access to more than 60 state-of-the-art solvers in more than a dozen optimization categories. The NEOS Server offers a variety of interfaces for accessing the solvers, and jobs run on distributed high-performance machines enabled by the HTCondor software. The NEOS Server is available free of charge for everyone, anywhere in the world.
  2. How can I get my own copy of the solvers available on NEOS?
    Solvers available on the NEOS Server are not available for download from this site. Some of the solvers are commercial products, which are made available for use for free through the NEOS Server. Others are NEOS Server implementations of academic codes or open source codes, which may or may not be available for download. For information about a particular solver and its availability, check the solver's interface page on the NEOS Server site or the solver's information page on the NEOS Guide site. Also, you can do a web search on the name of the solver or the name(s) of the developer(s).
  3. What are the hardware specs for the NEOS Server machines?
    Most of the solvers are hosted at the University of Wisconsin in Madison but some are hosted at other institutions. For the solvers that are hosted at the University of Wisconsin in Madison, the hardware specs are listed below. Please note that jobs do not run on neos-1 and neos-6.
    • neos-2 and neos-4 are Dell PowerEdge R410 servers with the following configuration:
      • CPU - 2x Intel Xeon X5660 @ 2.8GHz (12 cores total), HT Enabled
      • Memory - 64GB RAM
      • Disk - 2x 500GB SATA drives setup in RAID1
      • Network - 1Gb/s Ethernet
    • neos-3 and neos-5 are Dell PowerEdge R420 servers with the following configuration:
      • CPU - 2x Intel Xeon E5-2430 @ 2.2GHz (12 cores total), HT Enabled
      • Memory - 64GB RAM
      • Disk - 2x 2TB SATA drives setup in RAID1
      • Network - 1Gb/s Ethernet
    • neos-7 is a Dell PowerEdge R430 server with the following configuration:
      • CPU - 2x Intel Xeon E5-2698 @ 2.3GHz (32 cores total), HT Enabled
      • Memory - 192GB RAM
      • Disk - 4x 300G SAS drives setup in RAID5
      • Network - 1Gb/s Ethernet
  4. Are there any papers that describe the NEOS Server?
    • Czyzyk, J., Mesnier, M. P., and Moré, J. J. 1998. The NEOS Server. IEEE Journal on Computational Science and Engineering 5(3), 68-75. This paper discusses the design and implementation of the NEOS Server.
    • Dolan, E. D. 2001. The NEOS Server 4.0 Administrative Guide. Technical Memorandum ANL/MCS-TM-250, Mathematics and Computer Science Division, Argonne National Laboratory. This technical report, which discusses the implementation of the server and its use in detail, is available for download in PDF format.
    • Dolan, E. D., Fourer, R., Moré, J. J., and Munson, T. S. 2002. The NEOS Server for Optimization: Version 4 and Beyond. Technical Memorandum ANL/MCS-P947-0202, Mathematics and Computer Science Division, Argonne National Laboratory. This preprint, available for download in PDF format, describes the advances in version 4 of the server.
    • Dolan, E. D., Fourer, R., Goux, J.-P., Munson, T. S. and Sarich, J. 2008. Kestrel: An Interface from Optimization Modeling Systems to the NEOS Server. INFORMS Journal on Computing 20(4), 525-538. This paper gives details of the design and use of the Kestrel interface to the NEOS server. It is available for download in PDF format.
    • Dolan, E. D. and Munson, T. S. 2001. The Kestrel interface to the NEOS Server. Technical Memorandum ANL/MCS-TM-248, Mathematics and Computer Science Division, Argonne National Laboratory. This user's guide to the Kestrel interface is available for download in compressed postscript format.
    • Ferris, M. C., Mesnier, M. P. and Moré, J. J. 2000. NEOS and Condor: Solving Nonlinear Optimization Problems over the Internet. ACM Transactions on Mathematical Software 26, 1-18. This paper discusses the use of Condor, a distributed resource management system, as a provider of computational resources for NEOS. The paper is available for download in PDF format.
    • Gropp, W. and Moré, J. J. 1997. Optimization Environments and the NEOS Server. Approximation Theory and Optimization, M. D. Buhmann and A. Iserles, eds., Cambridge University Press, pages 167-182. This paper discusses the NEOS Server as a problem-solving environment that simplifies the formulation of optimization problems and the access to computational resources.
  5. How should I cite the NEOS Server in a paper?
    Cite the three papers listed here.
  6. Can I add my solver to the NEOS Server?
    We are always interested in learning about new solvers that would extend the capabilities of the NEOS Server. To submit your solver for evaluation and consideration, contact us by email at support@neos-server.org.

Submitting jobs

  1. How do I use the NEOS Server?
    The User's Guide to the NEOS Server provides detailed instructions on how to submit jobs to the NEOS Server but here is a brief overview of the steps.
    • First, select an appropriate solver for your optimization problem. The available solvers are listed by category at https://neos-server.org/neos/solvers/index.html. If you have already have a written formulation of your model, your choice of solver may be somewhat limited by the input formats available for the solvers.
    • Second, for the solver that you have selected, choose an appropriate input format (e.g., modeling language, programming language, solver-specific input format) and prepare the model, data and additional (e.g., commands) files as specified.
    • Third, select a method of submission for your job: web interface, email, XML-RPC, or Kestrel. If you use the web interface, you can upload the model, data, and additional files as text files. If you use the email interface, you will need to create in plain text an XML document using the solver-specified template. If you plan to use XML-RPC or Kestrel, consult the documentation on the NEOS Guide site.
    • Finally, submit your job to the NEOS Server. When the NEOS Server receives the submission, it assigns a job number and a password and it queues the job for scheduling on one of the machines. When the job is received by a machine, the submission is unpacked, the driver processes the submission, and the optimization code executes.
  2. Can I submit a job to NEOS from my own program instead of using the web interface?
    Yes. NEOS runs an XML-RPC server to communicate with clients for submitting jobs and retrieving results. As of July 2016, the NEOS XML-RPC server will accept HTTPS submissions at the URL https://neos-server.org:3333. Please update your client programs with the new URL to avoid problems with submissions. HTTP submissions to the URL http://neos-server.org:3332 will be phased out. There are client libraries written in Java and Python; a client library in Javascript is under development. See the NEOS API page for more information.
  3. Is NEOS available for use with SolverStudio?
    Yes! SolverStudio is an add-in for Excel 2007 and later on Windows that allows you to build and solve optimization models in Excel using many optimization modeling languages. SolverStudio allows models built using AMPL and GAMS to be solved using the NEOS server. For more information, see StudioSolver & NEOS.
  4. What is the maximum size for input files?
    The NEOS Server cannot accept input files larger than 16 MB. The 16 MB limit applies to submissions via the web interface, email and XML-RPC.
  5. Can I submit compressed files to NEOS?
    You cannot submit a compressed file of your entire job submission but you can compress and submit the individual files (e.g., model, data, commands). The NEOS Server checks each file to see if it is compressed, so you can compress any data for which the interface has a Browse option. The NEOS Server supports .gz, .z and .zip.
  6. Can I use NEOS to read additional input files or write additional output files?
    For security reasons, jobs are not allowed to read additional input files after they have been received by NEOS, and GAMS jobs cannot do a $call or execute statement.

    In general, jobs are not allowed to write additional output files; one exception is that GAMS jobs are allowed to write a text file, specifically named results.txt. This output file can be retrieved from the NEOS Server in the html output directory for the job. See the FAQ about retrieving GDX output for details on locating the html directory.

    There are several other methods that can achieve a similar effect to writing an output file.

    If you have your own copy of AMPL or GAMS, you can have the modeling language software read your files locally before submitting your job to NEOS via the Kestrel interface. When the solver is finished, the results are returned to your local AMPL or GAMS environment where you can write whatever file(s) you need. More information about Kestrel is available on the NEOS Server site.

    If you are using the NEOS XML-RPC interface to submit your job to NEOS, you can write a simple program to retrieve the NEOS output and write it to a file. More information about writing client code for the XML-RPC interface is available in this section.

  7. Can I submit subroutines or data for my objective function in separate files?
    It depends on the solver. Some solvers require all of the subroutines to be in a single file, while others allow separate files. Consult the solver's interface page for specific details.
  8. Where can I get more information on the automatic differentiation tools used by the NEOS Server?
    The Fortran solvers use ADIFOR and the C solvers use ADIC or ADOL-C.

Checking the queue and managing jobs

  1. How do I check the solver queue?
    From the NEOS Server page, click on the View Job Queue and Job Results link to go to the job admin page. Then, select the View Job Queue button and click Submit.
  2. How do I terminate a job that is running or that is in the queue?
    From the NEOS Server page, click on the View Job Queue and Job Results link to go to the job admin page. Then, enter the job number and the job password, select the Kill or Dequeue Job button and click Submit.
  3. How long can my problem run using NEOS?
    By default, jobs submitted to the NEOS Server are assigned as long jobs, which means that they can run at most 8 hours. There are two important things to know about long jobs. First, results for long jobs do not stream. For information about how to retrieve results, see the answer to the question I submitted a job to NEOS but I never received the results. How can I retrieve the results? Second, no results are returned if the job times out. In some cases, the administrator of a solver will set a shorter maximum time; in other cases, an administrator of a solver will set a maximum number of iterations (the job will terminate at the iteration limit even if it is less than 8 hours). If you expect that your job will approach or exceed the 8 hour limit (or the iteration limit), you should use the solver's options to set a maximum time value less than 8 hours. That way, when the job terminates, you will receive a log of the progress made.

    Jobs submitted to the NEOS Server can be assigned as short jobs, which means that they can run at most 5 minutes. Use the <priority> tag to assign a priority to a job. Results for short jobs do stream.

    GAMS Users: GAMS sets a default limit of 1000 seconds but this limit can be extended by adding a "reslim" statement before the solve statement. For example, to increase the limit to 2000 seconds for a model with the name "my_model", you would include the statement my_model.reslim = 2000;

  4. What is the maximum amount of memory available for a job submitted to the NEOS Server?
    Jobs submitted to the NEOS Server are limited to 3 GB of RAM. No partial results are returned for jobs that terminate because they exceed the allowable memory limit. If your job terminates without returning any results, most likely it required too much memory.

    It is often with mixed-integer programming (MIP) problems that you would have issues with memory. MIP solvers accumulate and store in memory information about the branch-and-cut tree as they progress. At some point, the amount of memory required for the tree information may exceed the amount of memory available. Most MIP solvers offer a node file storage feature to store tree information in files in order to reduce the amount of memory needed.

    For Gurobi, you can use the parameters nodefiledir and nodefilestart to control the node file storage feature. For more information, see the documentation on the Gurobi website.

    For CPLEX, you can set the following parameters:

    set workmem 128
    set mip strategy file 2
    set mip limits treememory 16384
    

    This limits memory for nodefile to 128MB; if it starts getting bigger, it writes the nodes to a file. The last line sets the maximum size of the node file to 16GB.

Retrieving results

  1. I submitted a job to NEOS but I never received the results. How can I retrieve the results?
    If your job completed, you can view the results from the job admin page. From the NEOS Server page, click on the View Job Queue and Job Results link to go to the job admin page. Then, enter the job number and the job password, select the View Job Results button and click Submit.

    If you submit large and difficult problems, often you will receive results that only indicate the job was scheduled and started running, for example:

        MODEL.gms -limcol=0 -miqcp=cplex -suppress=0 -al=0 -ao=0 -limrow=0
        -threads=4 -rmiqcp=cplex -lf=gams.logfile -ps=0 -qcp=cplex -pw=79 -lo=2
        -execmode=2 -o=solve.out -cerr=10 -lp=cplex -mip=cplex -rmip=cplex
        -relpath=1
        Calling GAMS.
    
        File exists
        Executing on neos-7.neos-server.org
    

    The most common reasons that you would receive this result and no other result are that your job exceeded the time limit (8 hours) or the memory limit (3 GB). No partial results are returned for jobs that terminate due to the time or memory limit. See the question How long can my problem run using NEOS? or the question What is the maximum amount of memory available for a job submitted to the NEOS Server?. We are working to provide more meaningful error messages in the future.

  2. Why were no results streamed for my job when I submitted it with short priority?
    The amount of output that streams for a short priority job depends in part on the input method and the solver. For GAMS jobs, you must check the box "Return log file" for the solver to stream information. For AMPL jobs, you must specify output options for the solver using a commands file. Here are suggestions for some MILP solvers:
    • CPLEX: mipdisplay=1
    • Gurobi: outlev = 1 (default setting)
    • MOSEK: outlev = 2
    • XPRESS-MP: outlev=1
  3. NEOS returned the objective function value but how do I get the values of the variables?
    You need to specifically request that the solver return the values of the decision variables. The format of the request differs by solver and by input, so you will need to refer to the solver's documentation for specifics. For AMPL, see the answer to the question How do I get AMPL to return the values of the decision variables?
  4. Why hasn't the server responded to my emailed job yet?
    If you have not received any response from the server within a reasonable amount of time, you probably will not. Upon receipt of your job request, the NEOS Server should send you an immediate message with the job number of that request. Failure is often attributed to one of these causes:
    • The server will only respond to messages sent To: neos@neos-server.org. Command line mailers should only be used with the option to include recipients on a To: line in the message.
    • The message must be type text/plain, the simplest form of email message.
    • Your mail program may be pre- or post-appending information to your message. To avoid information appended at the end, add the END-SERVER-INPUT tag to your data.

    If the NEOS Server cannot parse the first line correctly, you should receive mail from the server with the offending line and the entire message below. If the NEOS Server is not responding at all to your email submissions, submit a test problem (choose one of the sample submissions for the solver) to see if the server can understand your mail program.

AMPL topics

  1. Where can I find answers to my AMPL questions?
    There are a number of resources available on the AMPL website. There is an AMPL FAQ as well as information and documentation for the solvers that work with AMPL.
  2. How do I include my model and data files in the AMPL commands? I am getting messages like: mymodel.mod not found.
    The NEOS Server renames the model and data files that are submitted, so you will get an error if you try to reference the name of the model file or the data file in an AMPL command. NEOS automatically loads the model and data files, so it is not necessary to write include statements in your AMPL command file.
  3. What does AMPL: Variable is not defined mean?
    If you have include or reset commands in your AMPL commands, you are likely to get an error message:
    Executing AMPL.
    Error:
    processing data.
    Error: amplin, line XXXX (offset XXX):
    processing commands.
    Error: XXXX is not defined
    No variables declared.

    This error arises because NEOS does some preprocessing on the submitted model, data and command file. It concatenates everything into a single file and adds some options, such as specifying the solver, before sending it to be scheduled. Certain commands, particularly include and reset cause problems during the preprocessing. To resolve the error, remove the command(s).
  4. When I am using SolverStudio AMPL with NEOS, why do I get a message that says my computer cannot connect with NEOS?
    This error indicates that your PC cannot connect to the NEOS Server, perhaps because of a company firewall or the policies on your PC blocking external access. You should talk to your network administrator; he/she may be interested in the Internet Access notes on the SolverStudio Download & Install page.
  5. When using Xpress-AMPL, what does File not found: _amplpb.sol mean?
    In general, it means that Xpress has encountered an error in the solution process. However, we have found that the main cause is usually that Xpress cannot find a feasible point. The error may look something like this:
    XPRESS-MP 12.13: maxtime=7000
    Error preparing solution file
    ? 85 Error: File not found: _amplpb.sol
    ?122 Error: Corrupt solution file.
    exit code 1
  6. How do I get AMPL to return the values of the decision variables?
    If you are using a solver that accepts command files, submit a file that contains a solve command followed by the commands to display results. For example, to create a table of all variables with their lower bounds, optimal values, and upper bounds, use the following commands:
    solve;
    display _varname, _var.lb, _var, _var.ub;

GAMS topics

  1. I checked the box for "Return GDX output" but it was not returned to me. How can I retrieve it?
    You can retrieve the output from the NEOS Server by going to the html output directory for your job. Jobs are grouped into directories by 10,000s. To get the output directory, round down the job number to the nearest 10,000. Then, go to
    https://neos-server.org/neos/jobs/directory_number/job_number.html

    At the bottom of the page, you will find the link(s) to the additional output. If the link is missing or broken, you can use the following specific URL where job_number is the job number and job_password is the associated password:

    https://neos-server.org/neos/jobs/directory_number/job_number-job_password-solver-output.zip
  2. When I view my results from View Job Queue and Job Results, the page stops loading before I can see the links to additional output. What is the problem?
    The problem may be that your output file is too large (too many lines). To suppress some of the output (e.g., not printing the constraint values), add commands to the top of the GAMS file. For example:
    option limrow = 0;
    option limcol = 0;
    option solprint = off;
    \$offlisting \$offsymxref offsymlist
  3. When I am using MINOS-GAMS, why do I get "Error cannot open parameter file"?
    If you are using MINOS-GAMS via NEOS, MINOS 5.51 is selected by default, and NEOS names the option file that you submit as minos.opt. Specifying option nlp=minos5; in your submission has the effect of requiring the option file to be named minos5.opt. The conflicting names, minos.opt versus minos5.opt, cause the error.
    *** Error Cannot open parameter file "/scratch/neos/solver/jobs/XXXXXXX/minos5.opt"
    *** Error Error code = 2; No such file or directory

    Remove the command option nlp=minos5 to resolve the problem.

Error messages

  1. I am trying to submit a job using the web submission interface. When I click on submit, why does the browser return a blank page?
    If you receive a blank screen after submitting a job, it almost always means that there are non-ASCII characters in your job file(s) or that there is a syntax error in the input file. The blank screen results because the NEOS Server cannot properly parse the XML file. Check your input for accented characters, Greek letters or mathematical symbols and delete or replace them. If you are having trouble finding where the characters are in the file, try saving it as a text file or opening and saving it in a plain-text editor.
  2. What do I do after a web submission if the initial web page stops loading and does not refresh to the results page?
    By default, the priority of a job is long; output for long jobs does not stream and jobs can run for at most 8 hours. When the job has completed, the results will be emailed to you, so always provide a valid email address. If you do not receive an email, you can go to the job admin page to view the queue or job results. Also, see the answers to the questions I submitted a job to NEOS but I never received the results. How can I retrieve the results? and How long can my problem run using NEOS?
  3. Why did my job stop running and report Time limit reached. Process killed.?
    Jobs submitted with the default priority of long can run for at most 8 hours. Jobs submitted with the priority short can run for at most 5 minutes. See the answer to the question How long can my problem run using NEOS? for more information.
  4. What do "Error: Not enough memory" and "Error: Too much memory used -- 3207366576 bytes; couldn't get 38400 more." mean?
    These errors mean that your job submission has reached/exceeded the maximum of 3GB of RAM.
  5. What do I do when the solver displays my submission as garbage?
    This error may be an indication that your job submission contained non-ASCII symbols from a word processing program or other program. Try saving the file as a text file to get rid of the formatting symbols.
  6. When I am using XML-RPC to connect to NEOS, why do I get a time out error?
    The time out error arises when NEOS receives too many connection requests within a short period of time. There are two options for your code: (1) submit a request for the final results and wait for them to return or (2) loop over the status of the job but place a wait in the loop so that it doesn't loop more than once every 5 seconds.

    This error may occur if you are using a third-party application to access NEOS. The issue has been resolved in OpenSolver 2.8.0, so please upgrade if you are using an older version.

  7. What does "ERROR: Runtime error encountered." mean?
    A run time error indicates that an error occurred during the solution process, often a syntax error or a floating point exception (overflow or subscript out of bounds). In most cases, the solver should detect the error and report a message about the problem. In rare cases, the solver may terminate unexpectedly with no error message; please contact us to let us know about the problem.
  8. Why do I get a page-not-found error when I click on the link to a solver?
    The NEOS Server undergoes periodic maintenance and is upgraded from time to time, so the page may have moved. Also, solver administrators may choose to disable a solver at any time. If the solver appears on the list of solvers but you cannot access it, please contact us.
  9. Is ADIFOR broken if gnu make has syntax error messages from Adifor.cpp?
    Not necessarily. Sometimes the code has trouble if the user data contains lines with only TAB characters in them. Check to make sure that your input files do not contain TABs, which can be added automatically by an editor.