DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. P/Invoke with C++ bool Return Values

P/Invoke with C++ bool Return Values

Sasha Goldshtein user avatar by
Sasha Goldshtein
·
Jan. 27, 13 · Interview
Like (0)
Save
Tweet
Share
5.80K Views

Join the DZone community and get the full member experience.

Join For Free

i encountered an interesting gotcha today, which i thought would be interesting to share with you. the symptoms were the following: i wrote a simple c++ function exported from a dll, and tried to invoke it using a p/invoke wrapper. the c++ function returns a bool , and was declared as follows (the code was simplified for expository purposes):

extern "c" __declspec(dllexport) bool isprime(int n) { 
    if (n <= 1) return false; 
    if (n == 2) return true; 
    if (n % 2 == 0) return false; 
    for (int i = 3; i < n; ++i) { 
        if (n % i == 0) return false; 
    } 
    return true; 
}

the managed counterpart was:

[dllimport(@"..\..\..\debug\nativedll.dll", 
           callingconvention=callingconvention.cdecl)] 
static extern bool isprime(int n);

at runtime, for some values of n , the output would be inconsistent. specifically, when n =0 or n =1, the managed wrapper would return true – although the c++ function clearly returns false . for other values, however, the wrapper returned the right values.

next, i disassembled the c++ function, to find the following code responsible for the first branch:

cmp dword ptr[n], 1 
jg branchnottaken 
xor al, al 
jmp bailout 
… 
bailout: 
… 
ret

in other words, to return false , the function clears out the al register (which is the lowest byte of the eax register), and then returns. what of the rest of the eax register? in the debug build, it’s likely to contain 0xcccccccc, because of this part of the function’s prologue:

lea edi,[ebp-0cch]  
mov ecx,33h  
mov eax,0cccccccch  
rep stos dword ptr es:[edi]

to conclude, the function returns 0xcccccc00 in the eax register, which is then interpreted as true by the managed wrapper. why? because by default, p/invoke considers bool parameters to be four-byte values, akin to the win32 bool , and then any non-zero value is mapped to true .

to fix this, i had to convince p/invoke to marshal back the return value as a single byte. there’s no dedicated type for that in the unmanagedtype enumeration, but u1 (unsigned byte) or i1 (signed byte) will do the trick:

[dllimport(@"..\..\..\debug\nativedll.dll", 
           callingconvention=callingconvention.cdecl)] 
[return: marshalas(unmanagedtype.u1)] 
static extern bool isprime(int n);


i am posting short links and updates on twitter as well as on this blog. you can follow me: @goldshtn

Data Types twitter Debug (command) Build (game engine) Clear (Unix) Links IT Blog Branch (computer science)

Published at DZone with permission of Sasha Goldshtein, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Simulating and Troubleshooting StackOverflowError in Kotlin
  • How to Cut the Release Inspection Time From 4 Days to 4 Hours
  • When AI Strengthens Good Old Chatbots: A Brief History of Conversational AI
  • Application Assessment Questions for Migration Projects

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: