View Issue Details

IDProjectCategoryView StatusLast Update
0032074FPCCompilerpublic2019-06-10 13:15
ReporterGeorg Hieber Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
PlatformavrOSembedded 
Product Version3.1.1 
Summary0032074: [FEATURE REQUEST] avr: typed constants (including string constants) residing in ROM
DescriptionToday, typed constants (and this includes all string constants, even if not declared as a typed consant) are transferred from flash ROM into RAM at startup. This facilitates their access (exactly as variables), and is in accordance with the documented behaviour that their value can be changed at run time.

This implementation has a serious drawback: it uses up a lot of, especially in the smaller AVR's, precious RAM. For that reason, an implementation that leaves typed constants in ROM and only accesses what is necessary (maybe copies on demand into a buffer) would be highly desirable. The behaviour of typed constants could either be controlled by the $J compiler directive - {$J+} enables the present situation that constants are copied into RAM and writeable at run time, {$J-} leaves them in ROM, or by a qualifier to the constant declaration like

    const greeting = 'welcome to my latest gadget'; progmem;

The latter would imply J- for the so qualified constant.
Additional InformationPresently it is not even possible to implement something similar in Assembler (if there is, I did not find it.). There is no way to insert anything that is not code into the ROM. Constants, Strings are copied, and the inline assembler does not accept the .byte, .ascii …. directives.

Just to demonstrate the importance: An application might have 50 short messages that can be shown on a LCD display, of an average length of 10 characters. Together these strings occupy 550 bytes. The user can select between English, German, French, Spanish and Italian. The program shall run on an atmega324, which has 1k of RAM...
TagsAVR
Fixed in Revision
FPCOldBugId
FPCTarget
Attached Files

Relationships

related to 0032479 new Loading all strings from flash to RAM at startup 

Activities

Thaddy de Koning

2017-06-28 22:32

reporter   ~0101381

Last edited: 2017-06-28 22:38

View 6 revisions

If these strings are actually in ROM, use absolute. I use that a lot.
Absolute works with an address and the ROM is within addressable space.

No need to do that in assembler. This is already (always has been) possible in pure pascal.
example:
var
 X:Integer absolute $40;

In other words: FPC already has the feature you requested. You just did not know about it?

Thaddy de Koning

2017-06-29 09:01

reporter   ~0101383

Last edited: 2017-06-29 23:19

View 3 revisions

Note that if you use my suggestion regarding avrstring = type string[31] you can calculate the string offset from the start of the string table.
Even overlay it with an array[0..laststring-1] of avrstring

Thaddy de Koning

2017-06-30 21:33

reporter   ~0101394

Georg, if my answer was not clear enough, plz ask for demo code.
Also ask to close this.

Georg Hieber

2017-07-06 19:59

reporter   ~0101554

Thaddy, my concern is, that the startup code >>automatically<< copies the whole ROM - block of typed constants into RAM. This behaviour is what I would like to have changed. At present, it is impossible to declare anything residing in ROM, except code, and not having it duplicated to RAM at startup.

I don't see, what absolute variable declarations have to do with that. If we just misunderstand each other, georg at hieber-stgt dot de.

Jeppe Johansen

2017-07-06 20:17

developer   ~0101556

I don't think this can be easily supported. Loading from program memory on AVR requires a separate instruction (E)LPM.

Georg Hieber

2017-07-11 02:55

reporter   ~0101656

Jeppe, I am of course aware of that. That's why I suggested "maybe copies on demand into a buffer". Just thinking aloud, a (comiler-) procedure that accepts three parameters, a pointer in ROM, a pointer in RAM and a length.

For the beginning it would already be helpful if the inline assembler would accept the .byte, .ascii etc. directives. Or, put these constants into ROM, and suppress their automatic copying at startup and let them only be accessible through dedicated procedures like

   procedure GetRom(var s: shortstring; length: byte; adr: pointer);
   procedure GetRom(var w: word; adr: pointer);

and so on. They would be quite easily implemented in assembler and encapsulate the lmm stuff. I would be happy to provide them.

Christo Crause

2017-07-12 10:07

reporter   ~0101670

I support this functionality. A short C-based discussion of why one would want to do this: http://www.embedds.com/store-constants-and-arrays-in-program-memory/

There are two routes used by GCC - named address spaces using a __flash qualifier (https://gcc.gnu.org/onlinedocs/gcc/Named-Address-Spaces.html#AVR-Named-Address-Spaces)

or the progmem attribute (https://gcc.gnu.org/onlinedocs/gcc/AVR-Variable-Attributes.html).

Not sure which option would be easier to fit into the current FPC design, or perhaps a new concept?

Simon Ameis

2017-10-02 18:12

reporter   ~0103155

There is also a discussion in the German Lazarus Forum on this topic: http://www.lazarusforum.de/viewtopic.php?f=10&t=11068

The {$J+/-} compiler switch would be a great idea. The program may use different access methods to the data depending of it beeing a real constant or a variable.

Christo Crause

2017-10-03 07:09

reporter   ~0103160

{$J+/-} controls whether assignment statements to const are accepted by the compiler. To specify the storage location and generate appropriate access to that storage seems a bit different to this.

Georg mention the modifier progmem, which is similar to what is used in GCC. Another possible location for data is EEPROM, which in GCC is specified by EEMEM attribute.

Examples of a possible convention for specifying non-RAM storage locations:

const
  MenuStr1 = 'Button1"; progmem;
  MCU_ID = 101; eeprom;

Of course EEPROM is writable, so the eeprom modifier may also be used for writable variables:

var
  sensor1_cal: byte; eeprom;

I therefore think there is an additional need for a modifier/attribute to indicate storage location. The compiler should then generate the appropriate section information and access code specific to the memory location.

Florian

2017-10-03 13:36

administrator   ~0103166

Doesn't do

const
  a : string = '';section '.eeprom';
  b : string = '';section '.text';


what you need?

Of course, you cannot use such variables directly in pascal code, because the compiler does not know how to access them properly and it is impossible to support such variables seamlessly in pascal code. But with separate Get* routines, it should be no problem.

Thaddy de Koning

2017-10-03 18:38

reporter   ~0103173

Last edited: 2017-10-03 18:47

View 6 revisions

Afaik - depending on address space and maybe OS - (eep)rom based memory IS accessible for read without loading everything into RAM. As long as it is mapped.
1984.. Commodore 64 and ALL modern androids... (lest you forgot) And afaik fpc respects memory boundaries? This can't possibly be a FPC limitation. Look elsewhere? I think it is already there...

Thaddy de Koning

2017-10-03 18:50

reporter   ~0103174

Last edited: 2017-10-03 18:51

View 2 revisions

E.g. On a Raspberry Pi you can map HW (rom/soc) address space without anything else by using a variable that is mapped to a physical address...

Christo Crause

2017-10-03 20:18

reporter   ~0103175

Not sure if the following is a completely correct explanation, but here goes:

Afaik on AVR accessing RAM, flash & EEPROM requires different instructions:
RAM - use LD(D) or LDS instructions
flash - use (E)LPM (except on tinyAVR architecture) instruction
EEPROM - use EEPROM address/data/control registers

So using absolute to specify a hardware address will not work to access flash or EEPROM address spaces - it will only work for RAM.

Christo Crause

2017-10-03 20:20

reporter   ~0103176

Florian, your suggested syntax is what I believe should happen inside the compiler anyway. I thought that a single modifier (progmem/flash, eeprom etc.) would look neater in user code, similar to Georg's original suggestion...

Florian

2017-10-03 20:32

administrator   ~0103177

@Christo: Yes, this can be easily done. The remaining question is, if there is need for further compiler support. Of course, in the sense that it can be implemented.

Timm Thaler

2017-10-03 23:59

reporter   ~0103181

Last edited: 2017-10-04 00:00

View 2 revisions

In AVR Ada the declaration looks like this:
text : AVR_String(1 .. 16) := "MyStringConstant";
pragma Linker_Section (text, ".progmem");

The string is used by:
   sptr : Program_Address := text'Address;
   tchr := Programspace.Get(sptr);

This compiles to:
...
 60c: 24 91 lpm r18, Z+ ;load from programm memory
 60e: 29 93 st Y+, r18 ;store to flash
...

It is also possible to put numerical values like lookup tables, sensor characteristics, response curves to the flash.

Christo Crause

2017-10-04 07:26

reporter   ~0103183

@Florian, not sure what exactly needs to be added to the compiler. Here is what I think the processing flow could be:
- mark a constant as residing in special storage (lets say progmem & eeprom to try and create a generalized solution)
- if special storage is read-only then make sure the const is not modified {J-}
- code generator generates specific access instructions for storage type: (e)lpm for progmem
- for progmem modifier, specify .section .progmem in generated assembler code so that linker puts data in correct location

Of course it would be ideal if the pascal implementation selected can also cover similar usage cases for other embedded families, but I am not familiar with other embedded families.

Note that eeprom access can be done in FPC in theory (I haven't tested), but without compiler specific support the user will have to implement the needed routines.

Timm Thaler

2017-10-04 13:08

reporter   ~0103187

Eeprom Access kan be done by writing und reading the eeprom registers like you write or read io-registers or timer registers. Eeprom is possible.

With flash access you also have to prevent the compiler from copying the strings to the ram at the beginning of the program. At the moment all string constants are stored in flash and copied to ram, and there you can address it like any other array.

Christo Crause

2017-10-04 21:41

reporter   ~0103190

Last edited: 2017-10-04 21:42

View 2 revisions

Apologies Florian, I now see (via the German Lazarus forum) that your code actually kind of works. I never knew about the section modifier in FPC.

This solves the first problem of specifying the location to the compiler. Next problem is how to correctly read this location. At the moment reading a value marked with section '.progmem' generates an LDS instruction which reads from RAM:

const test: byte = 127; section '.progmem';
0000009e <TC_sPsPROGMEMTEST_ss_TEST>:
  9e: 7f 00 .word 0x007f ; ????

000000a0 <PASCALMAIN>:
var blank: byte;

begin
  a0: 0e 94 58 00 call 0xb0 ; 0xb0 <FPC_INIT_FUNC_TABLE>
  blank := test;
  a4: 00 90 9e 00 lds r0, 0x009E ; << correct flash address, but LDS reads from RAM
  a8: 00 92 00 01 sts 0x0100, r0
end.

At least the data is accessible from inline assembler, the following test works for avr5:

program progmemtest;

const test: byte = 127; section '.progmem';

var blank: byte;

begin
  asm
    ldi r30, lo8(test)
    ldi r31, hi8(test)
    lpm r0, Z
    sts blank, r0
  end;

  if blank = 127 then
    blank := 42;
end.

Florian

2017-10-04 22:54

administrator   ~0103191

Yes, together with a routine to load from prog. memory, this should work already reasonable well. Maybe George can provide those routines as promised :)

> At the moment all string constants are stored in flash and copied to ram, and
> there you can address it like any other array.

As said, this can be easily overcome with the section directive.

Timm Thaler

2017-10-04 23:17

reporter   ~0103194

Yes I tried and it works with inline assembler to get the data.

Christo Crause

2017-10-05 21:09

reporter   ~0103207

There is still the issue of error checking, for instance the following assignment should be (and is) allowed:

var x: byte; section '.eeprom';
...
x := 10;

But the following should not be allowed by the compiler (but is), unless called from a boot loader:

var x: byte; section '.progmem';
...
x := 10;

I would really like the compiler to help with error checking as much as possible when writing Pascal code, even for these corner cases.

kupferstecher

2017-10-07 18:22

reporter   ~0103237

Last edited: 2017-10-07 18:24

View 2 revisions

With regard to the feature request, I think its important to implement some further compiler support. Each nontrivial program uses constants and on target embedded nearly all constants should be used in the way that RAM is not occupied. Thus usage should be as easy as possible. For easier implementation it may be limited to String, Integer and Float, though. Furthermore implementing it for constants should be enough. Writing the Eeprom or Flash is a much more special case, so using the already working concept does its job.

(Edit: The same functionalities are required for target ARM embedded.)

Using constants should work as easy as this:

 const
   AOffset: Byte = 42; PROGMEM;
   ErrText: string = 'Error1'; PROGMEM;

 begin
   SensorVal := RawVal + AOffset;
   SerialOut(ErrText);

Florian

2017-10-15 19:21

administrator   ~0103450

@kupferstecher: This is not possible in general. Especially SerialOut(ErrText); is not "supportable", as it requires most likely that the address of ErrText can be taken and a data pointer is passed to SerialOut. However, getting a data pointer for data in the prog. mem is not possible. Using an implicit buffer might have other undesired side effect as it is in general not possible to determine what happens with an address.

kupferstecher

2017-10-27 19:01

reporter   ~0103799

Last edited: 2017-10-27 19:25

View 3 revisions

@Florian: I thought to easy on this topic.

My idea was to define a function returning the string on the stack:
Function GetProgMem_String(ProgMemString: ShortString): ShortString;

(The inside of such a function probably is quite difficult and done in assembler. The string argument is passed by reference, I think. Then the code can fetch the string from the ProgMem, size information is also there and return it as function result.
Regarding the example, this would result in such a line:
SerialOut( GetProgMem_String(ErrText));

In statements the predefined function could be added by the compiler depending on the data type, resulting in discussed syntax.

 const ErrText: string = 'Error1'; PROGMEM;
 begin
   SerialOut(ErrText);
)



The result of the below function is passed throught the reference to the memory space of 'AString', right? So in the AVR case 'AString' would be copied to RAM on programm initialisation. Thats where I made the mistake...

Function ReturnString():ShortString;
begin
  Result:= 'AString';
end;

Issue History

Date Modified Username Field Change
2017-06-28 15:57 Georg Hieber New Issue
2017-06-28 22:32 Thaddy de Koning Note Added: 0101381
2017-06-28 22:33 Thaddy de Koning Note Edited: 0101381 View Revisions
2017-06-28 22:34 Thaddy de Koning Note Edited: 0101381 View Revisions
2017-06-28 22:35 Thaddy de Koning Note Edited: 0101381 View Revisions
2017-06-28 22:38 Thaddy de Koning Note Edited: 0101381 View Revisions
2017-06-28 22:38 Thaddy de Koning Note Edited: 0101381 View Revisions
2017-06-29 09:01 Thaddy de Koning Note Added: 0101383
2017-06-29 23:19 Thaddy de Koning Note Edited: 0101383 View Revisions
2017-06-29 23:19 Thaddy de Koning Note Edited: 0101383 View Revisions
2017-06-30 21:33 Thaddy de Koning Note Added: 0101394
2017-07-06 19:59 Georg Hieber Note Added: 0101554
2017-07-06 20:17 Jeppe Johansen Note Added: 0101556
2017-07-11 02:55 Georg Hieber Note Added: 0101656
2017-07-12 10:07 Christo Crause Note Added: 0101670
2017-10-02 18:12 Simon Ameis Note Added: 0103155
2017-10-03 07:09 Christo Crause Note Added: 0103160
2017-10-03 13:36 Florian Note Added: 0103166
2017-10-03 13:37 Florian Relationship added related to 0032479
2017-10-03 18:38 Thaddy de Koning Note Added: 0103173
2017-10-03 18:38 Thaddy de Koning Note Edited: 0103173 View Revisions
2017-10-03 18:42 Thaddy de Koning Note Edited: 0103173 View Revisions
2017-10-03 18:42 Thaddy de Koning Note Edited: 0103173 View Revisions
2017-10-03 18:46 Thaddy de Koning Note Edited: 0103173 View Revisions
2017-10-03 18:47 Thaddy de Koning Note Edited: 0103173 View Revisions
2017-10-03 18:50 Thaddy de Koning Note Added: 0103174
2017-10-03 18:51 Thaddy de Koning Note Edited: 0103174 View Revisions
2017-10-03 20:18 Christo Crause Note Added: 0103175
2017-10-03 20:20 Christo Crause Note Added: 0103176
2017-10-03 20:32 Florian Note Added: 0103177
2017-10-03 23:59 Timm Thaler Note Added: 0103181
2017-10-04 00:00 Timm Thaler Note Edited: 0103181 View Revisions
2017-10-04 07:26 Christo Crause Note Added: 0103183
2017-10-04 13:08 Timm Thaler Note Added: 0103187
2017-10-04 21:41 Christo Crause Note Added: 0103190
2017-10-04 21:42 Christo Crause Note Edited: 0103190 View Revisions
2017-10-04 22:54 Florian Note Added: 0103191
2017-10-04 23:17 Timm Thaler Note Added: 0103194
2017-10-05 21:09 Christo Crause Note Added: 0103207
2017-10-07 18:22 kupferstecher Note Added: 0103237
2017-10-07 18:24 kupferstecher Note Edited: 0103237 View Revisions
2017-10-15 19:21 Florian Note Added: 0103450
2017-10-27 19:01 kupferstecher Note Added: 0103799
2017-10-27 19:12 kupferstecher Note Edited: 0103799 View Revisions
2017-10-27 19:25 kupferstecher Note Edited: 0103799 View Revisions
2019-06-10 13:15 Dimitrios Chr. Ioannidis Tag Attached: AVR