Compress and Decompress Blocks
Last updated
Last updated
Download and open the COMPRESSION.tscript script in the Script Editor (Lab - Scripts - Load from file)
The script makes it possible to work with several timeframes at the same time. Let's say that in the Source we have a timeframe of 1 minute, and we can receive data from several timeframes when using several compression blocks, (for example, 15 minutes, an hour, 33 minutes).
For example, we can receive data and calculate the signal on the hourly or on other timeframes, and place orders with a timeframe of 1 second.
The Compress block is very simple. It builds a larger specified interval from the bars received from the source.
In the block, the first parameter is Interval. You need to know about it that it can only be a multiple of the interval in the Source. If the Source has 5 minutes, then the Compress block should have 5, 10, 15, 20, etc. minutes. And accordingly, it cannot be less than in the Source.
If a time interval of 1 minute is selected in the Source, then in the Compress block the interval can be equal to or greater than 1 minute.
In a simple example of the Donchian channel algorithm, the source is set to 1 minute, and the indicators are calculated on an hourly interval. We output indicators from the compression block, in which the interval is one hour, and the script is recalculated and places orders with an interval of one minute.
In a simple Compress block, all bars start from the beginning of the loaded history (from the beginning of the chart) and if you have a standard timeframe multiple of 24 hours (or, for example, an hour, as in the above example), then you can use a simple Compress block. 15 minutes, 30 minutes, all intervals will be drawn exactly, without displacement.
If you need a non-standard timeframe, for example 53 minutes, then you may need to align at a specific hour, say exactly 10am.
The problem itself, why this may be needed, lies in the peculiarities of the block's operation.
As a rule, it is important for us that the compressed bars match in the script and the agent. If we keep the entire history in the script, then in the agent, as a rule, we restrict it for a faster recalculation of the agent. As a result, the last bars may not match in the script and the agent, which leads to a difference in calculations between the script and the trading agent.
To solve this problem, you need to use the Compress (Advanced) block, which can be configured so that compression draws bars not from the beginning of history, but from a certain hour in a day, or, for example, from the beginning of each hour, if a non-standard timeframe is used.
Let's look at a few illustrations to understand the settings of the advanced compress block.
Let's take a closer look at the problem: a non-standard timeframe without adjustment, compressed bars at the same hour on different days have different times. Let's take a look at a complessed bar around twenty one hours every day.
In the block settings, we have set 53 minutes, the adjustment is not set. Thus, the bars are drawn simply from the beginning of the history.
If you set the adjustment to 1440 minutes, i.e. one day, now every 24 hours the program starts drawing bars from zero hours and zero minutes.
Now, if you look at the chart of a bar every day, for example, at 21:00, the beginning of the non-standard timeframe is always the same, that is, we have enabled the adjustment of drawing bars to the beginning of the day. This binding to 00 hours is always valid for determining the block parameters.
The Adjustment shift parameter is used. If you set the Shift parameter equal to 600 (600 minutes -> 10 hours), then the Compression is equalized every day at exactly 10:00, and not at 12 at night.
The block allows you to do adjustments every hour. If you set 11 minutes in the Interval, and Adjustment every hour, then every hour Compression draws a new compressed bar. At the end of the hour there is always an incomplete bar.
Another example to better understand the compression logic and look at one more parameter. Consider the same 11 minutes, but the adjustment will be done every three hours, with the condition that the first bar should start at exactly 10 am.
In this case, we use Shift (shift always works from the last adjustment). If you look at the bars, they started to appear:
at 00:00 hours;
at 03:00 am;
at 06:00 am;
at 09:00 am.
So, between the beginning of the formation at 9 am and 10 am there are 5 minutes left. It is on it that we moved the drawing of the bars to the right.
A shift in a simple Compression block does everything the same, but from the beginning of drawing bars. In a simple Compression block, it may be needed, for example, if the session starts not at 10 am, but at 10.30 am. Old blocks in the program can be changed, but only if they do not affect the scripts that users have already built using these blocks. Therefore, as a rule, it happens in the program that a new block with a new name appears, but the old one remains.
It should be noted here, if it has not yet become clear why there are no bars at 00:00 hours? The answer is simple - there were no trades in the market. If there were trades, the bars would be drawn. Therefore, adjustment is unlikely to be needed to test the instruments of exchanges operating 24/7.
Decompression in the program works automatically if compressed (data from the Compress block) and uncompressed data (from the Source) are encountered in calculations within the same expression or chart.
Exception with the Updating value block (automatically decompressed).
That is, the compressed interval is always automatically converted to the source interval in case two different streams are used in a common computation.
The decompression is highly dependent on the calculation order.
The formation of the script is not determined by the actual use of data from another block in the calculation expression, but simply by the presence of a connection.
In the example below, even if OpenM1 is not used in the expression, all the same, these expressions will be automatically decompressed, since there is an incoming uncompressed stream.
If there are two different intervals in the block, then both of them will be decompressed, converted automatically to the base interval of the source.
Download and open the Decompress.tscript script in the Script Editor (Lab - Scripts - Load from file)
If you need to get any indicators and calculations in a reduced form for the base timeframe in the source, you need to calculate before decompression what needs to be obtained within this Compression and use only the final calculation for Decompression.
For example, if you need to use EMA, we fully calculate it and only output the final result in Decompression, after which it can be used with data from the source or Decompressions from other intervals, other Compressions.
For examples with a boolean expression or any other logic inside compression/decompression, it is also necessary to perform calculations immediately before decompression, and only the decompression itself must be used in computation blocks together with data from the source.
The Decompress block has one single parameter, the actual decompression method for the compressed data. This parameter defines the time shift of the data calculated for the compressed time intervals during decompression to the original (base) time interval in the source..
For example:
The basic calculation interval is 1 minute. Compression with an interval of 1 hour was used to calculate the indicators.
The 12.00 hour bar starts at 12.00 hours, this will be the first 1-minute candlestick and at 12.59 the last 1-minute candlestick begins.
When the indicators are decompressed:
1st method: the value 12.00 starts working (it will become available for calculations within the script) from the close of the minute bar 12.59
2nd method: the value 12.00 starts working (it will become available for calculations within the script) from the close of the minute bar at 12.00.
3rd method: the value 12.00 starts working (it will become available for calculations within the script) from the close of the minute bar at 13.00.
The 2nd method is not recommended for testing strategies, because it will be "looking into the future", that is, the value of the compressed indicator will be available before the end of the hour!
The 2nd method is applicable exclusively for visual and graphical analysis. The 1st method is the most optimal for optimizing trading strategies.
The third method has a small delay equal to one base interval, i.e. one minute in our example, and can be used to test strategies based on short (seconds) intervals, to simulate possible execution delays, such as slow communication channels, etc. Plus, for low-liquid instruments, in order not to wait for the end of the base bar, you can use the Interval + trade recalculation (in the script properties) together with the third method. The signal will always be received exactly with the end of the big bar, as soon as the aggregate of the ticks of the end of the hour arrives.
The method used by the Decompress block is selected by default in the script properties:
The first compressed bar is duplicated due to the fact that there is no data before the close of the first Compressed interval. The opening price of the first bar is used.
Ignore the first compressed bar, start any analysis from the second compressed bar.
Two completely identical logical expressions produce different data.
In the original script, there are two absolutely identical blocks Bull_HandedDw and Bull_HandedDw1, which show different results. Why?
Visually, according to the graph, the AND block is not associated with Bull_HandedDw. Why is deleting AND affecting Bull_HandedDw. And most importantly, how the user should understand it.
In this case, the sequence of adding blocks to the editor affects.
One is added before the AND block, and the other after. The AND block needed to decompress the data. This means that the second block will be calculated with decompressed data, and the first with compressed.
That is, in order for this script to be calculated in the same way, you need to break the connection between the HammerUp and AND blocks through the Compress block.
In future versions, we will make flow indication according to the principle Compressed, Decompressed.
