Optimization

General system and computer requirements

The higher the number of processor threads, the better for optimization speed.

Select the amount of RAM based on the tasks and the calculation algorithm described below.

How much RAM is required for optimization?

The final memory consumption during optimization can be calculated using the following formula:

Memory (bytes) = Number_of_Bars * 8 * Number_of_Unique_Parameter_Sets

Number of Bars – how many history bars have you passed to the script
8 – the number of bytes that one double number occupies
Number of Unique Parameter Sets – the number of unique combinations of parameters in the script. This value is not always equal to the number of iterations in the script. For two moving averages with a period from 1 to 300, the number of unique sets will always be 299, while the number of recalculations will be much larger. And by caching these 299 unique results, the optimization speed gain will be obtained, which is achieved by using GetData.

One gigabyte is always 1024 megabytes, and one megabyte is 1024 kilobytes, and a kilobyte is always 1024 bytes.

Within the program, one double number can take up to 10 bytes. TSLab allocates memory not only for the cache, but also for other intermediate calculations, and the closing prices of the instrument are also cached (which may take up some amount of memory). Therefore, memory will be spent a little more than in the calculations according to the above formula.

What determines the speed of optimization and script recalculation?

The program fully supports multithreading. Therefore, the more threads the processor supports, the faster the optimization is, with a sufficient amount of RAM.

The faster (high frequency) the processor and the more cores (possible threads), the faster the optimization. Read the related article:

Insufficient load on a multicore processor

You can open the Script Optimization menu by selecting File -> Program Settings -> Script Optimization

For multi-core processors, enable the "Should be used on multiprocessor (or many cores) systems" option

Script debugging - this is a debugger for working with Visual Studio or SharpDevelop
Script execution timeout - the default is 40 seconds. This is the time to recalculate the script. If the computer was unable to calculate the script within the specified time, the program will stop recalculation and remove the task. The setting affects not only the optimization, but this parameter is the maximum time for calculating the agent.
Script cache size - is selected by the brute-force search method. All computers are different, even with the same hardware. Depending on the drivers, installed programs, OS version and running services, they may behave differently. With an excess of RAM and working in 64bit on the latest versions of Windows, this parameter can be left alone or set close to the available free physical RAM.

About the brute-force search method from the user:
RAM is engaged in storing data, from where it is consumed and recalculated by the processor. A larger amount of RAM will allow you to place a larger amount of calculated data there without the need to extract this data from the HDD. The processor is responsible for the recalculation itself, respectively, the optimization speed will be determined by the balance between the amount of RAM and the processor speed. Increasing the RAM without increasing the processor speed will not have an effect after passing the point of maximum RAM load, just as increasing the processor frequency will have a lesser effect with low RAM, because data will have to be taken from low-speed data storage devices such as HDD and SDD.
The "brute-force search method" in this case implies the need to find a balance between the amount of data that needs to be optimized, processor frequency, RAM size, HDD / SSD speed and the number of processes already running in your system.
This is the "brute-force search method". You can play Need For Speed on your computer while optimization is running. It is clear that, all other things being equal, optimization in this mode will be slower than on the machine on which the game is not running. It seems that this is exactly what was meant in the documentation for TSLab
Alexandr Ganov

Optimization threads number - 0 means that the program will use the number of threads based on the number of cores.In the example, the picture shows a regular dual-core 2ghz xeon. It has 2 threads by default, since there are two cores. However, if you use hyper-threading processors, for example i7, then the number of threads can be taken by the number of cores, multiplied by two. The optimization speed will increase significantly.
Stop opt. if RAM is not enough - the program will stop optimization if the RAM resources are exhausted. If you uncheck the flag, Windows can take processes to SWAP (when using 64 bits) or throw a memory exception error (when using 32 bits)
Should be used on multiprocessor (or many cores) systems - the setting is available since version 2.1. This is a NET garbage collector setting. For example, for multi-core xeon (more than 8 cores), configuration is required, even regardless of the Windows version (server or not). With fewer cores, the setting may not affect the processor load.

Optimization in the program

Optimization types

Brute force - Direct enumeration of parameters with a given step from smaller to larger, in turn.
Random - Enumeration of parameters with a given step, in random order
Genetic - Read more in the article API. Plugins. Implementing IOptimizationMethod

How to enable indicator parameters for optimization?

Optimization parameters are enabled in the block properties.

Number of optimization iterations

Let's set the boundaries of the parameter optimization, for example, 1 - 100, step = 1. In total, a maximum of 100 iterations.

But if you set 30 iterations, not the maximum (100), then the optimizer will go through not exactly 30 iterations, but 34-35.

Explanation:

"Number of iterations" specifies the minimum number of iterations. In general, the number should be a multiple of the number of parameters involved.

Optimization Results

The Optimization Results table appears at the beginning of the optimization process unless the Save to Excel option is selected on the Optimization page. The table has a column Result from script. In the toolbox, the editor has a Result for optimization block. The block code is here: https://github.com/tslab-hub/handlers

The block outputs the value from the Result for optimization script into a column by block name.

There can be several results for optimization (limited by common sense)

Nothing is counted in the column, i.e. the block simply outputs the value from the Result for optimization to the Optimization Results table. For example, some characteristics evaluating a strategy.

If the Optimization Results table does not load, check the filter settings:

Saving a set of parameters after optimization

Click the Save parameters button in the "Optimization" tab. Give a name to the parameter set. This set of parameters will be available in the "Parameters" tab

Queue of multiple optimizations

There are several quotes. It is necessary to run the script for these quotes with certain sets of optimizing parameters and save the results to a file. Is it possible to solve this problem by creating an additional script that will perform the above task? It would be very convenient to put the script at night so that all planned optimizations are performed at night. Since optimizations can take 2-3 hours, it would be convenient to schedule several calculations to run overnight. Which will significantly speed up the optimization process.

It is not possible to queue several optimization tasks with built-in tools in TSLab. We have plans to implement these opportunities in the future.

Calculating GPU optimizations (CUDA computing)

Will there be GPU optimization in the program? The GPU is not capable of doing all the calculations, and in this case it is technically impossible to do, so the answer is no.

Last updated 2 years ago