Shoutbox

What?

The structure definition class allows developers to easily define, create, access and modify memory structures to use in many parts of the Win32 API. Instead of calculating the memory positions of the individual members and doing all the needed conversion by hand, this class does the hard work for you and lets you access members by name.

Why?

The power of this class really shows itself with an example. Say you needed a POINT structure to retrieve the current mouse position with GetCursorPos. According to MSDN, a POINT is defined as followed:

cpp code:

---

typedef struct tagPOINT {
  LONG x;
  LONG y;
} POINT, *PPOINT;

---

Using just the Plus! scripting engine, we could retrieve the cursor position with:

js code:

---

var pt = Interop.Allocate(8);
Interop.Call( 'user32', 'GetCursorPos', pt );
var x = pt.ReadDWORD(0);
var y = pt.ReadDWORD(4);

---

For small structures like this one, this is still pretty manageable. However when you need a structure with 10 or 20 members with different types and sizes, it quickly becomes frustrating to figure out the right positions.

Now have a look at how this same task can be achieved with this class:

js code:

---

var POINT = new StructType({
    x: DataType.LONG,
    y: DataType.LONG
};
var pt = new POINT;
Interop.Call( 'user32', 'GetCursorPos', pt.dataptr() );
var x = pt.x();
var y = pt.y();

---

As you can see, this is much easier to read and to work with. If that doesn't convince you, try a more complex example:

js code:

---

var LVITEM = new StructType({
    mask:            DataType.UINT,
    iItem:            DataType.INT,
    iSubItem:        DataType.INT,
    state:            DataType.UINT,
    stateMask:        DataType.UINT,
    pszText:        DataType.LPTSTR,
    cchTextMax:        DataType.INT,
    iImage:            DataType.INT,
    lParam:            DataType.LONG,
    iIndent:        DataType.INT,
    iGroupId:        DataType.INT,
    cColumns:        DataType.UINT,
    puColumns:        DataType.UINT
});

var firstItem = new LVITEM;
firstItem.mask( /* LVIF_TEXT */ 0x1 );
firstItem.pszText( 'First item' );

PlusWnd.SendControlMessage( 'MyListView', /* LVM_INSERTITEMW */ (0x1000 + 77), 0, firstItem.dataptr() );

---

Not only the memory positions are handled by the class, it also does the needed type conversions for you. In this example, the pszText member takes a string, creates a new DataBloc, writes the string to it and then writes the pointer of that DataBloc to the structure.

Features

• Define once, use forever. By defining a structure type, you create a new class which creates structures. You can create as many class instances as you want, so you don't have to redefine that type ever again.
  
• Defining structures is almost like copy pasting from MSDN. You can use the same names for the members and in almost all cases, you can use the same names for the data types as well. If MSDN says that 'mask' is a LONG, then just define it as a DataType.LONG, no need to figure out the corresponding DataBloc method any more.
  
• Accessing members couldn't be easier: just use their names. No more fiddling with memory positions, the class always knows where the members are located.
  
• Combined getter/setter methods for each member. To write a value to a member, pass it as the first parameter. To read the value of a member, just pass no arguments. Can it be any easier?
  

  js code:

  ---

  // Get
  var x = struct.myMember();
  // Set
  struct.myMember(10);

  ---

• Convert values automatically to the right type. Some DataTypes have a custom setter methods which allow you to pass a value in different types, such a string for an LPSZ, an [x,y] array for a POINT or red/green/blue values for a COLORREF.
  
• Use structures in structures. Occasionally, you'll find that a structure contains another structure. For example, NMLISTVIEW has a NMHDR structure for its 'hdr' member. Instead of copying the members of NMHDR to NMLISTVIEW, just create an NMHDR type and use it within NMLISTVIEW:
  

  js code:

  ---

  var NMHDR = new StructType({
      hwndFrom:    DataType.HWND,
      idFrom:        DataType.UINT,
      code:        DataType.UINT
  });
  var NMLISTVIEW = new StructType({
      hdr:        NMHDR,
      iItem:        DataType.INT,
      iSubItem:    DataType.INT,
      uNewState:    DataType.UINT,
      uOldState:    DataType.UINT,
      uChanged:    DataType.UINT,
      ptAction:    DataType.POINT,
      lParam:        DataType.LONG
  });

  ---

  You can then access a sub structure member with:
  

  js code:

  ---

  var nmListView = new NMLISTVIEW;
  var hwndFrom = nmListView.hdr().hwndFrom();

  ---

  This is possible since StructTypes double as DataTypes, so you can use them just like you'd use a DataType. (prototypical inheritance ftw!)
  
• Easily work with datatype arrays. Some structures take an array of values for a single member, for example LOGFONT takes an array of 32 TCHARS for its 'lfFaceName' member. Instead of defining 32 different TCHAR members in a structure type, you can use DataType.Array to create an array-like data type with a fixed length.
  

  js code:

  ---

  var LF_FACESIZE = 32;
  var LOGFONT = new StructType({
      lfHeight:    DataType.LONG,
      // ...
      lfFaceName:    DataType.Array(DataType.TCHAR, LF_FACESIZE)
  });
  var font = new LOGFONT;
  // Write a single item to a specific index
  font.lfFaceName().item(0, 'a');
  // Read a single item from a specific index
  var char = font.lfFaceName().item(0);
  // Write an array
  font.lfFaceName().fromArray(['a', 'b', 'c']);
  // Read as an array
  var chars = font.lfFaceName().toArray();
  // (W)CHAR arrays even have extra methods for easier handling
  font.lfFaceName().fromString('abcdef');
  var fontFace = font.lfFaceName().toString();

  ---

Download

» Structure definition class (v1.0.001 - 7 November 2010)

Change-log

• 1.0.001 (7 November 2010)
  
  • Initial release.

To-do

• More DataTypes? Post your suggestions below!

Note
This thread was split from SmokingCookie's thread about list-view groups. I threw in the structure class while trying to get grouping support in my ListView class. Because this was the first time the community heard about this class, there was a lot of excitement about it so the thread got split. That's why some of the first replies here may refer to this other thread.

quote:

---

Originally posted by Matti
I spend some time today trying to get this to work, and eventually I got it working.

---

Wow Matti - that's something amazing! Get it released soon Can you first tell us what DataType's you support?

At the moment, I implemented some basic types (BOOL, WORD, INT, CHAR, LPWSTR) and some common synonyms (UINT, DWORD, LONG). I also added a POINT type which basically writes an array [x, y] to two INTs.

To define a DataType, you just need to specify its size in bytes and a getter and setter method. For example, INT is defined with size 4 and two methods which simply map to ReadDWORD and WriteDWORD. For LPWSTR, the getter method uses some functions from kernel32 to get the string length of the pointer, then copy its contents and read it. The setter simply creates a DataBloc storing the given string and writes the pointer to that block.

This can still be extended upon though, I'm thinking about adding the possibility to use other StructTypes as DataTypes. For example, a NMLISTVIEW structure has an NMHEADER structure for its first 16 bytes. Instead of redefining the NMHEADER members in the NMLISTVIEW definition, it can be easier to just use NMHEADER as the data type of the first member hdr.

Ultimately, you should be able to just copy-paste the structure definitons from MSDN and rewrite it to produce a valid JavaScript hash with keys and data types. I'm not there yet, but even in this early stage it made my life much easier while working on the new ListView class!

You should add this by default

http://msdn.microsoft.com/en-us/library/aa383751(VS.85).aspx

Wow, looks like I've missed something in the past 36 hours :P

@Matti: it might come in handy to have something like StructureType.AddDataType(); that accepts a type name, a size and how to write it into the DataBloc obect (ie: using writeDWORD(); or writeSTRING();), or to send a callback function along with it that receives the position and data to write.

quote:

---

Originally posted by matty
You should add this by default

http://msdn.microsoft.com/en-us/library/aa383751(VS.85).aspx

---

Did you see how long that list is? I'll be more than happy if I could implement the most commonly used types, and it still won't be enough.

quote:

---

Originally posted by SmokingCookie
Wow, looks like I've missed something in the past 36 hours

@Matti: it might come in handy to have something like StructureType.AddDataType(); that accepts a type name, a size and how to write it into the DataBloc obect (ie: using writeDWORD(); or writeSTRING(), or to send a callback function along with it that receives the position and data to write.

---

That's exactly how I did it. The following comes straight from my implementation:

js code:

---

DataType.INT = new DataType({
    size : 4,
    read : function( databloc, position ) {
        return databloc.ReadDWORD( position );
    },
    write : function ( databloc, position, value ) {
        databloc.WriteDWORD( position, (1*value) & 0xFFFFFFFF );
    }
});

---

The DWORD implementation maps to the INT functions but also removes the sign from the numbers, so -2 becomes 0xFFFF FFFE.

On a totally unrelated note then... While I was playing with these signed/unsigned types, I saw that the code member from NMHDR was defined as a UINT, but could also take signed values (e.g. LVN_COLUMNCLICK = -108) so to get around this, I defined code as an INT instead. Anyone has a valid explanation for this behavior? Source: NMHDR on MSDN

It's quite funny how similar your struct system is to one I started making

How do you define elements that are arrays with your system? e.g. lfFaceName in LOGFONT.
I was trying to find a clean way of doing it, but I gave up (unlike me! )

Here's an example for what I did for strings:

js code:

---

API.typedef('string', function(length){
    return {
        size : ((length + 1) * 2),
        get  : function(dataBloc, offset) {
            return dataBloc.ReadString(offset);
        },
        set  : function(dataBloc, offset, value) {
            dataBloc.WriteString(offset, value);
        }
    }
});

---

But I "built" the DataBloc during construction rather than with an extra function, so it might be easier to do variable-size members your way.

It's nice to know you made almost exactly the same thing anyway, it means I don't need to finish mine now

quote:

---

Originally posted by Eljay
But I "built" the DataBloc during construction rather than with an extra function, so it might be easier to do variable-size members your way.

---

Actually, that build() method simply returns the DataBloc created during construction. When I started working on it, I first laid out the structure and I was still unsure about how I was going to implement it. I kept the build() method so the actual DataBloc cannot be accidentally replaced or removed from the class instance. (Well, you can still destroy the DataBloc with firstItem.build().Size = 0 but I don't think any sane developer would do that).

As for the arrays of a data type - great idea, thanks for your suggestion! I think you'll understand that working with fields with fixed sizes is much easier to work with than variable sized fields. Still, it may be possible to let the size be defined when the structure type is assigned to a structure member. You could add a constructor to the data type which takes a parameter (e.g. the amount of TCHARs) and sets its size accordingly. A LOGFONT could then look like this:

js code:

---

var LOGFONT = new StructType({
    // ...
    lfFaceName: new DataType.TCHAR(32)
});

---

In general, any DataType could take an array length as parameter and morph into an array type, such as INT[10] or LONG[5]. In the TCHAR constructor, this general behavior could then be extended to also allow a 32-character string to be accepted for the getter and to be outputted by the setter method by making use of inheritance.

I'm just thinking out loud here, the exact syntax may vary from what I really make but that's one way to do it.

The signed/unsigned behaviour and handling was the first thing I was wondering about when I saw you made a class for this, Matti. I was wondering if you took that into account... (since the WriteDWORD function can accept signed values too, yet you make it possible to define stuff like UINT, etc)... but I didn't checked your code yet to see for myself if you took that into account (lazy )... Anyways, since you brought it up...

quote:

---

Originally posted by Matti
While I was playing with these signed/unsigned types, I saw that the code member from NMHDR was defined as a UINT, but could also take signed values (e.g. LVN_COLUMNCLICK = -108) so to get around this, I defined code as an INT instead. Anyone has a valid explanation for this behavior? Source: NMHDR on MSDN

---

A signed value can always be represented as an unsigned value and vice versa. So if you assign -108 to an UINT (=4 bytes in memory), you actually assign the unsigned value: 0xFFFFFF94 or 4294967188 in memory.

And -108 is just easier to remember and to recognize in code than 4294967188, that's all. Actually -108 (LVN_COLUMNCLICK ) comes from the calculation LVN_FIRST-8, and LVN_FIRST is defined as 0xFFFFFF9C (or -100 when you interpret is as a signed INT).

Thus, code should actually be defined as UINT though, not as an INT! So, instead of that 'workaround' by defining it as a wrong type, your function should convert the given signed value to an unsigned value, which is actually automatically already done by WriteDWORD anyways. That is: it simply writes the given number as-is to memory, which is already enough... (after of course AND'ing it with 0xFFFFFFFF to avoid overflow errors).

But to read the UINT value again, you must do some calculation yourself though. ReadDWORD() will otherwise return -108 (in the above example). But since it is defined as UINT it should return 4294967188 instead. So, what you should do is check the return value and if <0 you need to add 0x100000000 to the returned value.

In short (taking INT/UINT as example):
- To write a value you don't need to do anything (accept AND'ing with 0xFFFFFFFF of course). It doesn't matter if the number is pos or neg or if the type is signed or not...
- But to properly return an unsigned value, you do need to check if the returned value isn't negative, if so, you add 0xFFFFFFFF+1 to it)

--

PS: Another thing you could come across which would be very related to this is that quite a lot of functions in Windows actually compare your assigned value with an OR function to a mask which represents the meaningfull bits. So, it could well be that a -108 value (to take the same number as above) is simply interpreted as 4 in case the mask with meaningfull bits is 0xF. So, in that case -108, -236, 292, 1104936484, -4294966732, etc all have the same function since the least significant digit is always 0x4.

In the other way around: there are also some types used in Windows which are defined as a signed type, but in fact only unsigned (positive) values are defined. That is: only some positive values have a meaning. This is in most cases by design. And in those cases, in later Windows versions, you'll see that some other values are added and sometimes they are indeed negative values. So, it is not because the type is signed and only positive values are defined now, that in a later Windows version some new (negative) values aren't added anymore.

I hope I make some sense here, it's already late and I didn't slept much last nights either...

Thanks for the very thorough explanation, Cookie! I already had such conversions implemented in my getters, I just needed to do the signed to unsigned conversion on LVN_COLUMNCLICK as well to make my comparison work. For ease of use, I exposed these methods in Number.toUnsigned(), Number.toSigned() and Number.toUINT(). Then again, it's a bit counter-intuitive to have DataBloc.ReadWORD() and ReadDWORD() take signed numbers while WORD and DWORD are both unsigned...

Since this thread is now separated from the list-view groups topic, I updated the first post with some details about the class. This should give you a better idea on what's coming.