2106 | ADC | ARM7 | Atmel | Bootloader | CAN | CrossStudio | CrossWorks | DDS | ECos | Ethernet | ETM | FIFO | FLASH | FPGA | GCC | GDB | GNU | GNUARM | GPIO | I2C | IAP | IAR | JTAG | Kickstart | LCD | Linux | LPC | LPC-E2294 | LPC2000 | LPC2100 | LPC2104 | Lpc2106 | Lpc210x | LPC2114 | LPC2119 | LPC2124 | LPC2129 | Lpc2138 | LPC213x | LPC21xx | LPC2210 | LPC2212 | LPC2214 | LPC2292 | LPC2294 | LPC2xxx | LPC3128 | MCB2100 | Olimex | Philips | PWM | Rowley | RTC | RTOS | SPI | SSP | UART | UART0 | UART1 | ULINK | USB | Watchdog | Wiggler

Hi, I'm having a little problem with the malloc function. As for what I have read trying to solve this, my guess is that I have not defined the heap properly or something like that.

I'm using an LPC2368 without any operating system or anything, just plain C. After "succesfully" reserving memory with malloc (as in not getting a null pointer), writing the block with zeros and printing them to check, I often find many bytes writen with some other data, just after I theoretically reserved and wrote them.

Keil help gives two alternatives to define a heap, each one related to diferent sizes of heap and speed of algorithms, but I could not fully understand how to set that up and I cannot sort this out with a static array, since my application has a lot of dinamic messages going to and from a device controlled by the processor.

I know this is an old problem, but I could not find a clear solution looking around... :S

Resuming, the question would be how to set up my code for malloc to work properly and avoid memory leaks?

Any help will be apretiated! Thanks,

Nic
------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Hi Nic,

what does "theoretically reserved and wrote them" mean? I think this is sort of basic and you should be able to find out who writes
what bytes where.

Anyway, I suggest you don't use malloc at all. If it is about messages, you might write your own BinaryMessageQueue.

How to avoid memory leaks? I'd suggest something intelligent like: the memory leaks and buffer overflows are forbidden in our
company... No, there is no easy answer in C, you just need to write correct code, and for that you have to understand the lifetime
of your "objects".

With regards,
Jan

----- Original Message -----
From:
To:
Sent: Tuesday, November 03, 2009 3:16 PM
Subject: [lpc2000] malloc and heap
> Hi, I'm having a little problem with the malloc function. As for what I have read trying to solve this, my guess is that I have
> not defined the heap properly or something like that.
>
> I'm using an LPC2368 without any operating system or anything, just plain C. After "succesfully" reserving memory with malloc (as
> in not getting a null pointer), writing the block with zeros and printing them to check, I often find many bytes writen with some
> other data, just after I theoretically reserved and wrote them.
>
> Keil help gives two alternatives to define a heap, each one related to diferent sizes of heap and speed of algorithms, but I could
> not fully understand how to set that up and I cannot sort this out with a static array, since my application has a lot of dinamic
> messages going to and from a device controlled by the processor.
>
> I know this is an old problem, but I could not find a clear solution looking around... :S
>
> Resuming, the question would be how to set up my code for malloc to work properly and avoid memory leaks?
>
> Any help will be apretiated! Thanks,
>
> Nic
>

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

At 05:58 PM 11/3/2009, you wrote:
>Hi Nic,
>
>what does "theoretically reserved and wrote them" mean? I think this
>is sort of basic and you should be able to find out who writes
>what bytes where.
>
>Anyway, I suggest you don't use malloc at all. If it is about
>messages, you might write your own BinaryMessageQueue.
>
>How to avoid memory leaks? I'd suggest something intelligent like:
>the memory leaks and buffer overflows are forbidden in our
>company... No, there is no easy answer in C, you just need to write
>correct code, and for that you have to understand the lifetime
>of your "objects".
>
>With regards,
>Jan

One thing that helps a little bit is to allocate local memory to the
function that way when you exit the function the allocation on the
stack is released automatically.
Cecil
k5nwa
www.softrockradio.org www.qrpradio.com
< http://parts.softrockradio.org/ >

Never take life seriously. Nobody gets out alive anyway.

[Non-text portions of this message have been removed]

------------------------------------

______________________________
Stellaris® MCU Family: New Parts, New Package, New Price.


(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Nic,
The functions, malloc and free can be used in an embedded design, but
with great care in their use. We are using them in our network stack code
and some other places. They are good way to get more memory out of a system
that has very little. We are not using Keils tools with the LPC23xx. We use
Rowleys Crossworks, they write very good tools and are a third the cost. So
I cannot help you with the Keil setups.

Make sure your size of heap is set to 1.5 times the amount of memory you
what to allocate. The heap has some hidden memory use, so never set it to
the exact size. We use 8K for ours. There was not a mention of "free" calls
in your post. For every malloc there needs to be a free, when the memory is
not needed anymore. One safe thing to do, is to free each allocation in the
reverse order they were malloc, just like a stack. This will go a long way
to reducing heap fragmentation. Over time this can cause a system crash. One
other good thing to do is, always check for NULL on malloc and make sure
your code knows what to do if it happens. And NEVER EVER free a NULL
pointer, this is bad, very bad. We also have a monitor running on our
systems that has a heap(and stacks) checker routine, this is a good way to
make sure you have no memory leaks.

As Jan said, if you are using this for messages you should not use
malloc and free for this. Lookup byte ques and write up one or Google some
code to do this. I believe your code may have other problems than heap
issues. You may be walking off of an array somewhere else in your code, or
have a stack overflow. You should check for these things too.
Hope this helps,
Michael Freeman
Principal Design Engineer
Update Systems, Inc.

-----Original Message-----
From: l...@yahoogroups.com [mailto:l...@yahoogroups.com]On Behalf Of
Jan Vanek
Sent: Tuesday, November 03, 2009 5:58 PM
To: l...@yahoogroups.com
Subject: Re: [lpc2000] malloc and heap
Hi Nic,

what does "theoretically reserved and wrote them" mean? I think this is
sort of basic and you should be able to find out who writes
what bytes where.

Anyway, I suggest you don't use malloc at all. If it is about messages,
you might write your own BinaryMessageQueue.

How to avoid memory leaks? I'd suggest something intelligent like: the
memory leaks and buffer overflows are forbidden in our
company... No, there is no easy answer in C, you just need to write
correct code, and for that you have to understand the lifetime
of your "objects".

With regards,
Jan

----- Original Message -----
From:
To:
Sent: Tuesday, November 03, 2009 3:16 PM
Subject: [lpc2000] malloc and heap

> Hi, I'm having a little problem with the malloc function. As for what I
have read trying to solve this, my guess is that I have
> not defined the heap properly or something like that.
>
> I'm using an LPC2368 without any operating system or anything, just
plain C. After "succesfully" reserving memory with malloc (as
> in not getting a null pointer), writing the block with zeros and
printing them to check, I often find many bytes writen with some
> other data, just after I theoretically reserved and wrote them.
>
> Keil help gives two alternatives to define a heap, each one related to
diferent sizes of heap and speed of algorithms, but I could
> not fully understand how to set that up and I cannot sort this out with
a static array, since my application has a lot of dinamic
> messages going to and from a device controlled by the processor.
>
> I know this is an old problem, but I could not find a clear solution
looking around... :S
>
> Resuming, the question would be how to set up my code for malloc to work
properly and avoid memory leaks?
>
> Any help will be apretiated! Thanks,
>
> Nic
>

[Non-text portions of this message have been removed]

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Hi,

Michael Freeman wrote:
> And NEVER EVER free a NULL pointer, this is bad, very bad.
I'm a little intrigued by why you say this. Could you elaborate on
why this is "bad, very bad" ..?

The C standard specifically states that you can safely pass a NULL
pointer to free() - 'safe' in that it is guaranteed to do nothing.

What I would strongly suggest is that immediately after the call to
free() you set the pointer to NULL. For example:

ptr = malloc( n );

...

free( ptr );
ptr = NULL;

That way if ptr escapes after the free() then any subsequent code
will be following a NULL pointer (easy to check/trap) rather than
something that *looks* like a normal pointer.

Cheers,
Neil
--
http://www.njohnson.co.uk

------------------------------------

______________________________
Stellaris® MCU Family: New Parts, New Package, New Price.


(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Okay your are right about free and NULL, but I still tell this to all
newbies. And heres why, this is only true for C99 compilers, it may not be
true of the others. In the embedded world there are still a lot of compilers
that are not 100% ANSI. In the past, I have burned good by those non ANSI
compilers. So, I write defensive code in the off chance it is used again on
a different target. Portable code starts with a lot of analness up front.

It is good programming to match all mallocs with frees, one to one, and to
understand they do not have a valid pointer. And to only use a pointer
within a checked state. If you write code this way you will not have memory
leaks, less dangling pointers(sometimes you miss a '*') and heap
fragmentation will be kept down. There are times where my strategy does not
work. More advance programming can be talked about some other time.

Sample code I like use to show this:

void* pvMyPointer;
....

if(pvMyPointer = malloc(2))
{ //Checked state
//do stuff
free(pvMyPointer);
}
else
{
//error handle
}
Lets look at the faster way to write this code. If you miss a set to
NULL(ptr = NULL), this will set you up for a dangling pointer aka wild
pointer. This is a very hard problem to chase down. If you miss the free
call(free( ptr )) you make a memory leak. And because you set the point to
NULL other code will think the pointer is invalid. This too is very hard to
find.

Neil's Code:

ptr = malloc( n );

...

free( ptr );
ptr = NULL;

Regards,
Michael Freeman
Principal Design Engineer
Update Systems, Inc.

-----Original Message-----
From: l...@yahoogroups.com [mailto:l...@yahoogroups.com]On Behalf Of
Neil Johnson
Sent: Wednesday, November 04, 2009 3:00 AM
To: l...@yahoogroups.com
Subject: Re: [lpc2000] malloc and heap
Hi,

Michael Freeman wrote:
> And NEVER EVER free a NULL pointer, this is bad, very bad.

I'm a little intrigued by why you say this. Could you elaborate on
why this is "bad, very bad" ..?

The C standard specifically states that you can safely pass a NULL
pointer to free() - 'safe' in that it is guaranteed to do nothing.

What I would strongly suggest is that immediately after the call to
free() you set the pointer to NULL. For example:

ptr = malloc( n );

...

free( ptr );
ptr = NULL;

That way if ptr escapes after the free() then any subsequent code
will be following a NULL pointer (easy to check/trap) rather than
something that *looks* like a normal pointer.

Cheers,
Neil
--
http://www.njohnson.co.uk

[Non-text portions of this message have been removed]

------------------------------------

______________________________
Stellaris® MCU Family: New Parts, New Package, New Price.


(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Hi,

Michael Freeman wrote:
> Okay your are right about free and NULL, but I still tell this to all
> newbies. And heres why, this is only true for C99 compilers, it may
> not be
> true of the others.

C89 compilers? Certainly the rationale for C89 specifically states
that free() is allowed to take a NULL pointer. Alas, I regret
throwing out my copy of C89 when I got a copy of C99, although I
still have a copy of the C89 rationale.

> In the embedded world there are still a lot of compilers
> that are not 100% ANSI. In the past, I have burned good by those
> non ANSI
> compilers. So, I write defensive code in the off chance it is used
> again on
> a different target. Portable code starts with a lot of analness up
> front.

Fair point.

> It is good programming to match all mallocs with frees, one to one,
> and to
> understand they do not have a valid pointer. And to only use a pointer
> within a checked state. If you write code this way you will not
> have memory
> leaks, less dangling pointers(sometimes you miss a '*') and heap
> fragmentation will be kept down. There are times where my strategy
> does not
> work. More advance programming can be talked about some other time.

Indeed.

> Sample code I like use to show this:
>
> void* pvMyPointer;
> ....
>
> if(pvMyPointer = malloc(2))
> { //Checked state
> //do stuff
> free(pvMyPointer);
> }
> else
> {
> //error handle
> }

Eessh. Assignment within an if() ... I would fail that in a code
review. Far too equally likely to be accidentally written

if(pvMyPointer == malloc(2))

which is valid C, but not what is intended.

> Lets look at the faster way to write this code. If you miss a set to
> NULL(ptr = NULL), this will set you up for a dangling pointer aka wild
> pointer. This is a very hard problem to chase down. If you miss the
> free
> call(free( ptr )) you make a memory leak. And because you set the
> point to
> NULL other code will think the pointer is invalid. This too is very
> hard to
> find.

Well, both ways are valid, and both ways have their issues. Its
either a personal choice, or one that is imposed by a coding standard.

Cheers,
Neil
--
http://www.njohnson.co.uk

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Well first of all, thank you all for answering! Mainly, I agree with both
Neil and Michael on what you propose, and would like to point out that
free() is being called properly after each message is treated. As for byte
queues, the problem is I'm actually integrating a satellite modem API into
my code, and despite I fully understand the way it behaves, it would imply =
a
mayor structural change and a rather stupid delay in development considerin=
g
the problem is triggered by a pretty much standard function as malloc(), so
I'm kinda trying to avoid going down that road...

Right now I have a mix of what Michael suggests, not passing Neil's code
review but still, working! :) and freeing the y
function B> called by a let's say function A. That is... to put an example:

if ( (ptr =3D malloc(x)) =3D=3D FALSE )
// report memory failure and exit...
else
{
// do stuff
free(ptr);
}

OR

// function A()
{
mySTRUCT structure;

B(&structure); --> B calls malloc() and assigns the pointer returned to a
field of 'structure'

free(structure.ptr);
}

I thought a lot about variables visibility (and more important, existence!)
but being as it is that the variable modified is defined in function A and
it's field set in a function B called by A, it should not be a problem.
Intermediate variables such as the size of the memory block requested might
disappear, but that information is already saved in the block's header if I
understand all this correctly...

Last, since the message treatment is fairly immediate, mallocs and
corresponding frees are called in sequence, so fragmentation should be kept
to a very minimum. I am not re-assigning NULL to the ptr after using it, bu=
t
I guess I'll change that... it seems like a good practice.

Now going back a little to the root of all this debate, my actual problem i=
s
defining the heap. I'm either not doing it right or not doing it at all! As
I mentioned, Keil help speaks of two ways to do so, but I'm not sure if thi=
s
will be taken care of when compiling (via project settings and the like) or
if I have to explicitly define/create it in my 'main.c' or wherever...

Oddly enough, the program works fine anyway, but I'm pretty sure something
is not properly setup, and considering this should run for many days I'd
like to solve the problem! My guess is that since the message size rarely
exceeds a few bytes, say 30, I'm being lucky (or actually not) and it works=
,
yet not right...

Thanks for the help so far, I truly appreciate it!

--=20
Nicol=E1s Christie
n...@gmail.com
[Non-text portions of this message have been removed]

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Okay Keil people, how do you setup a heap?

Getting to the point,
Michael Freeman

-----Original Message-----
From: l...@yahoogroups.com [mailto:l...@yahoogroups.com]On Behalf O=
f
Nico Christie
Sent: Wednesday, November 04, 2009 6:52 PM
To: l...@yahoogroups.com
Subject: Re: [lpc2000] malloc and heap

Well first of all, thank you all for answering! Mainly, I agree with both
Neil and Michael on what you propose, and would like to point out that
free() is being called properly after each message is treated. As for byt=
e
queues, the problem is I'm actually integrating a satellite modem API int=
o
my code, and despite I fully understand the way it behaves, it would impl=
y
a
mayor structural change and a rather stupid delay in development
considering
the problem is triggered by a pretty much standard function as malloc(),
so
I'm kinda trying to avoid going down that road...

Right now I have a mix of what Michael suggests, not passing Neil's code
review but still, working! :) and freeing the say
function B> called by a let's say function A. That is... to put an
example:

if ( (ptr =3D malloc(x)) =3D=3D FALSE )
// report memory failure and exit...
else
{
// do stuff
free(ptr);
}

OR

// function A()
{
mySTRUCT structure;

B(&structure); --> B calls malloc() and assigns the pointer returned to a
field of 'structure'

free(structure.ptr);
}

I thought a lot about variables visibility (and more important,
existence!)
but being as it is that the variable modified is defined in function A an=
d
it's field set in a function B called by A, it should not be a problem.
Intermediate variables such as the size of the memory block requested
might
disappear, but that information is already saved in the block's header if
I
understand all this correctly...

Last, since the message treatment is fairly immediate, mallocs and
corresponding frees are called in sequence, so fragmentation should be
kept
to a very minimum. I am not re-assigning NULL to the ptr after using it,
but
I guess I'll change that... it seems like a good practice.

Now going back a little to the root of all this debate, my actual problem
is
defining the heap. I'm either not doing it right or not doing it at all!
As
I mentioned, Keil help speaks of two ways to do so, but I'm not sure if
this
will be taken care of when compiling (via project settings and the like)
or
if I have to explicitly define/create it in my 'main.c' or wherever...

Oddly enough, the program works fine anyway, but I'm pretty sure somethin=
g
is not properly setup, and considering this should run for many days I'd
like to solve the problem! My guess is that since the message size rarely
exceeds a few bytes, say 30, I'm being lucky (or actually not) and it
works,
yet not right...

Thanks for the help so far, I truly appreciate it!

--
Nicol=E1s Christie
n...@gmail.com

[Non-text portions of this message have been removed]

=20=20
[Non-text portions of this message have been removed]

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

--- In l...@yahoogroups.com, Nico Christie wrote:

> Now going back a little to the root of all this debate, my actual problem is
> defining the heap. I'm either not doing it right or not doing it at all! As
> I mentioned, Keil help speaks of two ways to do so, but I'm not sure if this
> will be taken care of when compiling (via project settings and the like) or
> if I have to explicitly define/create it in my 'main.c' or wherever...
I read the Keil links yesterday and I thought they intended for you to define the heap and heap size and, in at least one case, the size had to be a power of 2. They don't really discuss actually allocating the space.

So, I downloaded FreeRTOS and there is an example of defining the heap in C:\FreeRTOS\FreeRTOS\Demo\ARM7_LPC2129_Keil_RVDS The space is actually allocated in Startup.s Note that the example doesn't actually use the heap but there is useful information in the file.

Maybe this will give you a starting point.

Richard

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

--- In l...@yahoogroups.com, "rtstofer" wrote:
>
> --- In l...@yahoogroups.com, Nico Christie wrote:
>
> > Now going back a little to the root of all this debate, my actual problem is
> > defining the heap. I'm either not doing it right or not doing it at all! As
> > I mentioned, Keil help speaks of two ways to do so, but I'm not sure if this
> > will be taken care of when compiling (via project settings and the like) or
> > if I have to explicitly define/create it in my 'main.c' or wherever...
I downloaded the Keil evaluation software. I opened a demo project C:\Keil\ARM\Boards\NXP\SJA2510\Blinky and when I have the Startup.s file open in the edit window, there is a configuration tab under the window. Here you can define the size of the heap and the .S file will automatically be updated.

Then if you add #include to the text and add some code using malloc() and free(), you will see in the .map file the address and size of the heap.

I didn't actually try to run any of the code. I assume the compiler is generating code for method #1. I didn't try adding a pragma either.

Richard

------------------------------------

______________________________
Stellaris® MCU Family: New Parts, New Package, New Price.


(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

Well I guess it was a dumb question :)
I've set up the heap, not yet fully tested for malloc to work properly
though...

Thank you Richard!

On Thu, Nov 5, 2009 at 12:55 AM, rtstofer wrote:

> --- In l...@yahoogroups.com , "rtstofer"
> wrote:
> >
> >
> >
> > --- In l...@yahoogroups.com , Nico
> Christie wrote:
> >
> > > Now going back a little to the root of all this debate, my actual
> problem is
> > > defining the heap. I'm either not doing it right or not doing it at
> all! As
> > > I mentioned, Keil help speaks of two ways to do so, but I'm not sure =
if
> this
> > > will be taken care of when compiling (via project settings and the
> like) or
> > > if I have to explicitly define/create it in my 'main.c' or wherever..=
.
>
> I downloaded the Keil evaluation software. I opened a demo project
> C:\Keil\ARM\Boards\NXP\SJA2510\Blinky and when I have the Startup.s file
> open in the edit window, there is a configuration tab under the window. H=
ere
> you can define the size of the heap and the .S file will automatically be
> updated.
>
> Then if you add #include to the text and add some code using
> malloc() and free(), you will see in the .map file the address and size o=
f
> the heap.
>
> I didn't actually try to run any of the code. I assume the compiler is
> generating code for method #1. I didn't try adding a pragma either.
>
> Richard
>
>=20=20
>

--=20
Nicol=E1s Christie
n...@gmail.com
[Non-text portions of this message have been removed]

------------------------------------

______________________________
Stellaris® MCU Family: New Parts, New Package, New Price.


(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )

--- In l...@yahoogroups.com, Nico Christie wrote:
>
> Well I guess it was a dumb question :)
> I've set up the heap, not yet fully tested for malloc to work properly
> though...
>
> Thank you Richard!

Not a dumb question at all!

I don't tend to use malloc() as it is hardly recommended for embedded programming with limited memory. But I might want to use it some day and so, it is worth learning about.

I had never used the Keil toolchain either. Again, a learning experience.

In most cases where I reply to a question, it first raises an opportunity to learn something. I try to find the answer for my own purposes and if I feel successful, I'll post a reply. If I am not successful, I'll wait for others to give me the answer.

Either way, I learn something for the time spent.

Richard

------------------------------------



(You need to be a member of lpc2000 -- send a blank email to lpc2000-subscribe@yahoogroups.com )