Here are some old reference articles, it's worth reading because it explains the root causes of all our problems:

• How to: Access DLLs in Excel
• And two older VB ones (VBA is an updated VB runtime): Passing Strings to a DLL Procedure Declaring a DLL Procedure

To sum up:

• VBA internal storage is BSTR with unicode characters in it.
• VBA also uses BSTR for talking with the external world, but you don't have to use BSTR if you don't want to because from C/C++, you may choose to use only the pointer part of the BSTR (a BSTR is a LPWSTR, an LPWSTR is not a BSTR).
• The content of the BSTR that VBA uses to communicate outside its world is not unicode but ANSI (VBA is still living in the 90s and thinks that, regarding the String data type, the outside world is always ANSI, ASCIIZ, CodePage, etc. ). So, even if it still uses a BSTR, that BSTR contains the ANSI equivalent of the internal Unicode storage, modulo the current locale (BSTR is like an envelope that can contain anything, including ANSI, including zero characters anywhere, provided the length matches the data).

So when you use use Declare with argument of type String, the final binary layout will always match C's ANSI 'char *' (or LPSTR in windows macro parlance). Officially, you're still supposed to use VARIANTs if you want to pass full unicode string over interop barriers (read the links for more on this).

But, not all is lost, as VBA (not VB) has been a bit improved over the years, mainly to support Office 64-bit versions.

The LongPtr data type has been introduced. It's a type that will be a signed 32 bit integer on a 32-bit system and a signed 64 bit integer on a 64-bit system.

Note it's the exact equivalent of .NET's IntPtr (VBA also still thinks a Long is 32-bit and an Integer is 16-bit, while .NET uses Long for 64-bit and Int for 32-bit...).

Now, LongPtr would be useless w/o the help of VB's all-time undocumented function StrPtr that takes a string and returns a LongPtr. It's undocumented because officially VB doesn't know what a pointer is (actually, be cautious as this can crash your program at runtime if not used properly).

So, let's suppose this C code:

  STDAPI ToUpperLPWSTR(LPCWSTR in, LPWSTR out, int cch)
  {
    // unicode version
    LCMapStringW(LOCALE_USER_DEFAULT, LCMAP_LINGUISTIC_CASING | LCMAP_UPPERCASE, in, lstrlenW(in), out, cch);
    return S_OK;
  }

  STDAPI ToUpperBSTR(BSTR in, BSTR out, int cch)
  {
    // unicode version
    // note the usage SysStringLen here. I can do it because it's a BSTR
    // and it's slightly faster than calling lstrlen...
    LCMapStringW(LOCALE_USER_DEFAULT, LCMAP_LINGUISTIC_CASING | LCMAP_UPPERCASE, in, SysStringLen(in), out, cch);
    return S_OK;
  }

  STDAPI ToUpperLPSTR(LPCSTR in, LPSTR out, int cch)
  {
    // ansi version
    LCMapStringA(LOCALE_USER_DEFAULT, LCMAP_LINGUISTIC_CASING | LCMAP_UPPERCASE, in, lstrlenA(in), out, cch);
    return S_OK;
  }

Then you can call it with these VBA declares (note this code is 32 and 64-bit compatible):

  Private Declare PtrSafe Function ToUpperLPWSTR Lib "foo.dll" (ByVal ins As LongPtr, ByVal out As LongPtr, ByVal cch As Long) As Long
  Private Declare PtrSafe Function ToUpperBSTR Lib "foo.dll" (ByVal ins As LongPtr, ByVal out As LongPtr, ByVal cch As Long) As Long
  Private Declare PtrSafe Function ToUpperLPSTR Lib "foo.dll" (ByVal ins As String, ByVal out As String, ByVal cch As Long) As Long

  Sub Button1_Click()

      Dim result As String
      result = String(256, 0)

      // note I use a special character 'é' to make sure it works
      // I can't use any unicode character because VBA's IDE has not been updated and does not suppport the
      // whole unicode range (internally it does, but you'll have to store the texts elsewhere, and load it as an opaque thing w/o the IDE involved)

      ToUpperLPWSTR StrPtr("héllo world"), StrPtr(result), 256
      MsgBox result
      ToUpperBSTR StrPtr("héllo world"), StrPtr(result), 256
      MsgBox result
      ToUpperLPSTR "héllo world", result, 256
      MsgBox result
  End Sub

They all work, however

• the ToUpperLPSTR is an ANSI fonction, so it will not support the unicode range that most people use nowadays. It works for me because the special non ASCII 'é' character coded in the IDE will find a correspondance when I run it in my machine with my ANSI codepage. But it may not work depending on where it runs. With unicode, you don't have that kind of problems.
• the ToUpperBSTR is kinda specialized for VBA (COM automation) clients. If this function is called from a C/C++ client, the C/C++ coder will have to create a BSTR to use it, so it may look funny and add more work. Note however it will support strings that contains the zero character in it, thanks to the way BSTR work. It can sometimes be useful for example to pass array of bytes or special strings.