Managed application suddenly fails to invoke DLLs

when you got a memory problem, it will hard to find the cause. We have seen failing scanners and failing GPRS connections for ‘no reason’. These cases where identified with a tool called memoRX available for WM5 only.

The problem with WM memory handling is, that all processes share the same DLL space. Every process is loaded into a 32MB memory slot. All DLLs loaded by any runnning process is ‘mirrored’ into these 32MB slots. Process memory is used from bottom to top and DLLs are loaded from top to bottom. When the usage borders get closer, strange problems may occur. Device.exe, which loads all drivers, is a critical memory slot.


bad memorx image:

      32 MB 02000000:  +--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+  32MB Top
            01f00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01e00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01d00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01c00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01b00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01a00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01900000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01800000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01700000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01600000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01500000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01400000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01300000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01200000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01100000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01000000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            00f00000:  |~~~~~~~~|~~~~~~~~|##~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
 Dll Low >> 00d80000:  |     ~~~|     ~~~|########|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~|     ~~~| << DLL Low: 0x00d80000
            00d00000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00c00000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00b00000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00a00000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00900000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00800000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00700000:  |        |        |########|        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00600000:  |        |        |########|        |        |....... |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00500000:  |        |........|########|        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00400000:  |        |........|########|        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00300000:  |        |........|########|.       |        |........|        |        |        |        |        |        |        |        |        |        |        |        |.       |
            00200000:  |        |........|########|........|......  |........|        |.....   |.       |.....   |        |        |        |        |        |        |        |...     |........|
            00100000:  |.       |........|########|........|........|........|......  |........|........|........|..      |......  |........|..      |.....   |..      |..      |........|........|
        0MB 00000000:  +--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+  0MB Bottom
                       |      NK| filesys|  device|  IQueue| shell32|    gwes| srvtrus| service| connmgr| SSClien|  calnot| AppCent| repllog| poutloo| IConnec|  memoRx| autorun|   cprog| DataSer|
               VM Used |00020000|004e1000|00e30000|00220000|001b0000|005cb000|000b0000|00190000|00120000|001a0000|00031000|000a2000|00100000|00031000|00091000|00033000|00031000|00140008|00210000|
             VM Remain |00d60000|0089f000|fff50000|00b60000|00bd0000|007b5000|00cd0000|00bf0000|00c60000|00be0000|00d4f000|00cde000|00c80000|00d4f000|00cef000|00d4d000|00d4f000|00c3fff8|00b70000|
               VM Base |c2000000|04000000|06000000|08000000|0a000000|0c000000|0e000000|10000000|12000000|14000000|16000000|18000000|1a000000|1c000000|1e000000|20000000|22000000|26000000|28000000|
                       +--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+

‘good’ memoRX image:

      32 MB 02000000:  +--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+  32MB Top
            01f00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01e00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01d00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01c00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01b00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01a00000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01900000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01800000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01700000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01600000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01500000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01400000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01300000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01200000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01100000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
            01000000:  |~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|~~~~~~~~|
 Dll Low >> 00ee0000:  |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        | << DLL Low: 0x00ee0000
            00e00000:  |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |        |
            00d00000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00c00000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00b00000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00a00000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00900000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00800000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00700000:  |        |        |........|        |        |        |        |        |        |        |        |        |        |        |        |        |
            00600000:  |        |        |........|....... |        |        |        |        |        |        |        |        |        |        |        |        |
            00500000:  |        |....... |........|........|        |        |        |        |        |        |        |        |        |        |        |        |
            00400000:  |        |........|........|........|        |        |        |        |        |        |        |        |        |        |        |        |
            00300000:  |        |........|........|........|        |..      |        |        |        |.       |        |        |        |        |        |.       |
            00200000:  |        |........|........|........|.....   |........|        |...     |.       |........|        |...     |        |        |        |........|
            00100000:  |.       |........|........|........|........|........|......  |........|........|........|..      |........|........|..      |.....   |........|
        0MB 00000000:  +--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+  0MB Bottom
                       |      NK| filesys|  device|    gwes| SSClien| shell32| srvtrus| service| connmgr|  IQueue|  memoRx|   cprog| repllog| poutloo| IConnec| DataSer|
               VM Used |00020000|004d1000|00cf0000|005d0000|001a0000|00223008|000b0000|00150000|00120000|00220000|00033000|00140008|00100000|00031000|00091000|00210000|
             VM Remain |00ec0000|00a0f000|001f0000|00910000|00d40000|00cbcff8|00e30000|00d90000|00dc0000|00cc0000|00ead000|00d9fff8|00de0000|00eaf000|00e4f000|00cd0000|
               VM Base |c2000000|04000000|06000000|08000000|0a000000|0c000000|0e000000|10000000|12000000|14000000|16000000|18000000|1a000000|1c000000|1e000000|28000000|
                       +--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+

So, although your app may not have a mem leak, it may fail to load a DLL suddenly, if another process has been loaded into mem and loaded additionaly DLLs in the meantime.

To get nearer to the source of the problem, you should log the running processes and create memory dumps, that show the running processes and the loaded DLLs. So you get a picture of the device memory usage in ‘normal’ and ‘worst’ cases.

Although there is a new tool called DevHealth and DevHealtherViewer, these will not really help to easily identify ‘bad’ memory situations as it is possible by using memoRX.

‘unknown’ memory status in devhealthviewer

DevHealthView screenshot

DevHealthView screenshot

If your apps uses self programmed DLLs, you should reduce a) the number of DLLs and b) rearrarange them to use nearly 64KB. Every loaded DLL will ‘waste’ a 64KB DLL slot. So four DLLs with 4, 8, 16 and 24 KB will consume 4x64KB=256KB DLL memory. If rearranged to one or two DLLs (ie 4+24 and 8+16) they will only occupy 128KB (2x64KB).

Another thing often overlooked is WM_HIBERNATE. When the OS gets into low mem situation it broadcasts a hibernate message. The apps have to ‘answer’ this message by reducing there memory use (dispose unused objects). If the overall mem does not change, the OS closes down processes by killing them to free up memory.

A good tool to stress test the app and the device is hoppeRX (search internet, it is free MS tool). It will simulated unordered screen taps and starts apps. MS states, that a device should run hoppeRX for 24 hours to be sure the device is running good at all situations.

If you get into the memory problem and cant shrink down your memory usage (DLL combining) and no other processes are started at the time of the failure, then contact the device manufactorers support and open a case. The engineers are able to supply you with more hints and maybe use devhealth dumps to analyse more.

~Josef

some links about memory:

http://bolingconsulting.com/blog/?m=200806

http://www.windowsfordevices.com/articles/AT7967807599.html

http://blogs.msdn.com/wmshell/archive/2006/05/25/607263.aspx

http://msdn.microsoft.com/en-us/library/ms836325.aspx

Leave a Reply