COCKOS
CONFEDERATED FORUMS
Cockos : REAPER : NINJAM : Forums
Forum Home : Register : FAQ : Members List : Search :
Old 08-12-2020, 03:23 PM   #1
stw
Human being with feelings
 
stw's Avatar
 
Join Date: Apr 2012
Posts: 279
Default uunicode string inputs

Hi,
i recently released my new plug and ran into some problems i haven't considered right before.
To unlock my plug the user has to input his name from which I calc a hash and some other stuff. I take the string from an ITextInputControl.
Xcode/mac properly translates all unicode characters into the corresponding bytes. Unfortunately VS/win does not. And so i get false hashs afterwards.
I asked already on stackoverflow and the answers give some hints. But...
Here a quote from my question:
Quote:
...
At the end i need to get a hash out of an user input string which must be the same on mac and win. After failing with all kind of approaches i started with just hashing (picosha2.h) a simple char. While the ASCII range hashs equal on both platforms e.g. const char* cc = "ø"; gives different results . Guessing from the error when changing declaration to unsigned is that Xcode treats the "ø" as const char[3] and VS as const char[2]. Another example "書" which spits out a correct unicode char array size of 4 in Xcode has still only 2 in VS.
If i look at the actual memory content i see that Xcode stores c3b8 for "ø" what matches the UTF-8 code, while VS shows a f8 (which would be the value for the Unicode Code Point for that letter which confuses me even more https://www.utf8-chartable.de/.)
Is there any chance to make VS interpret (runtime) strings as UTF-8 and store them the same way as Xcode does?
Adding "/utf-8" option in VS does change string interpretation but doesn't help with my ITextInputControl
I get the impression that it's an iplug/wdl issue on windows. Right now i'm really stuck and digging deeper into the windows functions stuff seems a bottomless pit. I hope to get some help here.

You're granted to get a perfect windows unicoded license key if you can help fixing that
stw is offline   Reply With Quote
Old 08-13-2020, 02:41 AM   #2
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,645
Default

I guess you need to use wide char versions of some Win32 API functions... What seems to work here is to use CreateWindowW(L"EDIT"), SetWindowLongPtrW(GWLP_WNDPROC), and SendMessageW(WM_GETTEXT), and then use WideCharToMultiByte(CP_UTF8, 0, wideCharBuf, -1, txt, sizeof(txt), NULL, NULL) to convert from wide char to UTF-8.

EDIT: Instead of SendMessageW(WM_GETTEXT) you could also add WDL/win32_utf8.c to your project and do GetWindowText().

Last edited by Tale; 08-13-2020 at 03:10 AM. Reason: win32_utf8
Tale is online now   Reply With Quote
Old 08-13-2020, 03:15 AM   #3
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,645
Default

BTW, an unrelated issue (with my IPlug version at least) is that IPlug doesn't seem to support international keyboards for some reason. I.e. if I type ' (single quote) + e, then usually I get é (e with acute accent), but in an IPlug text box this doesn't seem to work.

Note that this already doesn't work, even without wide char/UTF-8 support.

EDIT: Actually this works fine in VSTHost, so I guess this is a REAPER issue, not an IPlug issue.

Last edited by Tale; 08-13-2020 at 03:23 AM. Reason: REAPER issue
Tale is online now   Reply With Quote
Old 08-13-2020, 04:28 AM   #4
stw
Human being with feelings
 
stw's Avatar
 
Join Date: Apr 2012
Posts: 279
Default

Quote:
Originally Posted by Tale View Post
I guess you need to use wide char versions of some Win32 API functions... What seems to work here is to use CreateWindowW(L"EDIT"), SetWindowLongPtrW(GWLP_WNDPROC), and SendMessageW(WM_GETTEXT), and then use WideCharToMultiByte(CP_UTF8, 0, wideCharBuf, -1, txt, sizeof(txt), NULL, NULL) to convert from wide char to UTF-8.

EDIT: Instead of SendMessageW(WM_GETTEXT) you could also add WDL/win32_utf8.c to your project and do GetWindowText().
Hi Tale,
thanks for giving some hints. I tried to implement your suggestions but failed.
Starting with CreateWindowW(L"EDIT") had already a bunch of pitfalls.
E.g. CreateWindowW() wants a wchar_t* instead of a char* as second argument. I started to change all related function calls and finally ended with multiple
Quote:
Error C2440 'default argument': cannot convert from 'const char [1]' to 'const wchar_t *' ...\igraphics.h
I guess starting to make changes here will make things even worse, so i stopped. How do you get this to work?
stw is offline   Reply With Quote
Old 08-13-2020, 04:56 AM   #5
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,645
Default

Well, here is a patch on top of WDL-OL that should probably work:

Code:
diff --git a/WDL/IPlug/IGraphicsWin.cpp b/WDL/IPlug/IGraphicsWin.cpp
index 6c7dfcf2..4f2498ef 100644
--- a/WDL/IPlug/IGraphicsWin.cpp
+++ b/WDL/IPlug/IGraphicsWin.cpp
@@ -92,7 +92,9 @@ LRESULT CALLBACK IGraphicsWin::WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARA
           {
             case kCommit:
             {
-              SendMessage(pGraphics->mParamEditWnd, WM_GETTEXT, MAX_PARAM_LEN, (LPARAM) txt);
+              WCHAR buf[MAX_PARAM_LEN];
+              SendMessageW(pGraphics->mParamEditWnd, WM_GETTEXT, MAX_PARAM_LEN, (LPARAM) buf);
+              WideCharToMultiByte(CP_UTF8, 0, buf, -1, txt, sizeof(txt), NULL, NULL);
 
               if(pGraphics->mEdParam)
               {
@@ -964,7 +966,9 @@ void IGraphicsWin::CreateTextEntry(IControl* pControl, IText* pText, IRECT* pTex
     default:                  editStyle = ES_CENTER; break;
   }
 
-  mParamEditWnd = CreateWindow("EDIT", pString, ES_AUTOHSCROLL /*only works for left aligned text*/ | WS_CHILD | WS_VISIBLE | ES_MULTILINE | editStyle,
+  WCHAR buf[MAX_PARAM_LEN];
+  MultiByteToWideChar(CP_UTF8, 0, pString, -1, buf, MAX_PARAM_LEN);
+  mParamEditWnd = CreateWindowW(L"EDIT", buf, ES_AUTOHSCROLL /*only works for left aligned text*/ | WS_CHILD | WS_VISIBLE | ES_MULTILINE | editStyle,
                                pTextRect->L, pTextRect->T, pTextRect->W()+1, pTextRect->H()+1,
                                mPlugWnd, (HMENU) PARAM_EDIT_ID, mHInstance, 0);
 
@@ -976,7 +980,7 @@ void IGraphicsWin::CreateTextEntry(IControl* pControl, IText* pText, IRECT* pTex
 
   SetFocus(mParamEditWnd);
 
-  mDefEditProc = (WNDPROC) SetWindowLongPtr(mParamEditWnd, GWLP_WNDPROC, (LONG_PTR) ParamEditProc);
+  mDefEditProc = (WNDPROC) SetWindowLongPtrW(mParamEditWnd, GWLP_WNDPROC, (LONG_PTR) ParamEditProc);
   SetWindowLongPtr(mParamEditWnd, GWLP_USERDATA, 0xdeadf00b);
 
   //DeleteObject(font);
Do note that I don't really know up to what point you can mix ANSI/wide char, so maybe you will need to convert more or even all API calls to wide char.
Tale is online now   Reply With Quote
Old 08-13-2020, 06:26 AM   #6
stw
Human being with feelings
 
stw's Avatar
 
Join Date: Apr 2012
Posts: 279
Default

Quote:
Originally Posted by Tale View Post
Do note that I don't really know up to what point you can mix ANSI/wide char, so maybe you will need to convert more or even all API calls to wide char.
Wow... thanks a lot Tale. I'll try to implement that and see what happens.
Since i try to understand why macos and windows act so different here every background info is much appreciated. So if someone wants to shed a light on that, don't hesitate
stw is offline   Reply With Quote
Old 08-13-2020, 07:28 AM   #7
stw
Human being with feelings
 
stw's Avatar
 
Join Date: Apr 2012
Posts: 279
Default

Hey Tale,
you little genius. That did the job!!
Best service ever!! You qualified for unicode enabled license of my plug if you like

I never would have found that myself!

Best regards,
Stefan
stw is offline   Reply With Quote
Old 08-13-2020, 08:27 AM   #8
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,645
Default

Good to hear that is works!

I'm still not sure, but I think that because I'm using SetWindowLongPtrW(), ParamEditProc() is assumed to be wide char as well, which in turn means I should probably also use CallWindowProcW() and DefWindowProcW().

Anyway, here's an extended patch, replacing the one I posted earlier:

Code:
diff --git a/WDL/IPlug/IGraphicsWin.cpp b/WDL/IPlug/IGraphicsWin.cpp
index 6c7dfcf2..aabd7db5 100644
--- a/WDL/IPlug/IGraphicsWin.cpp
+++ b/WDL/IPlug/IGraphicsWin.cpp
@@ -92,7 +92,9 @@ LRESULT CALLBACK IGraphicsWin::WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARA
           {
             case kCommit:
             {
-              SendMessage(pGraphics->mParamEditWnd, WM_GETTEXT, MAX_PARAM_LEN, (LPARAM) txt);
+              WCHAR buf[MAX_PARAM_LEN];
+              SendMessageW(pGraphics->mParamEditWnd, WM_GETTEXT, MAX_PARAM_LEN, (LPARAM) buf);
+              WideCharToMultiByte(CP_UTF8, 0, buf, -1, txt, sizeof(txt), NULL, NULL);
 
               if(pGraphics->mEdParam)
               {
@@ -122,7 +124,7 @@ LRESULT CALLBACK IGraphicsWin::WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARA
             }
             case kCancel:
             {
-              SetWindowLongPtr(pGraphics->mParamEditWnd, GWLP_WNDPROC, (LPARAM) pGraphics->mDefEditProc);
+              SetWindowLongPtrW(pGraphics->mParamEditWnd, GWLP_WNDPROC, (LPARAM) pGraphics->mDefEditProc);
               DestroyWindow(pGraphics->mParamEditWnd);
               pGraphics->mParamEditWnd = 0;
               pGraphics->mEdParam = 0;
@@ -453,7 +455,7 @@ LRESULT CALLBACK IGraphicsWin::ParamEditProc(HWND hWnd, UINT msg, WPARAM wParam,
         if (pGraphics->mEdParam) break;
         LPARAM lres;
         // find out if the original control wants it
-        lres = CallWindowProc(pGraphics->mDefEditProc, hWnd, WM_GETDLGCODE, wParam, lParam);
+        lres = CallWindowProcW(pGraphics->mDefEditProc, hWnd, WM_GETDLGCODE, wParam, lParam);
         // add in that we want it if it is a return keydown
         if (lParam && ((MSG*)lParam)->message == WM_KEYDOWN  &&  wParam == VK_RETURN)
         {
@@ -478,9 +480,9 @@ LRESULT CALLBACK IGraphicsWin::ParamEditProc(HWND hWnd, UINT msg, WPARAM wParam,
         break;  // Else let the default proc handle it.
       }
     }
-    return CallWindowProc(pGraphics->mDefEditProc, hWnd, msg, wParam, lParam);
+    return CallWindowProcW(pGraphics->mDefEditProc, hWnd, msg, wParam, lParam);
   }
-  return DefWindowProc(hWnd, msg, wParam, lParam);
+  return DefWindowProcW(hWnd, msg, wParam, lParam);
 }
 
 IGraphicsWin::IGraphicsWin(IPlugBase* pPlug, int w, int h, int refreshFPS)
@@ -964,7 +966,9 @@ void IGraphicsWin::CreateTextEntry(IControl* pControl, IText* pText, IRECT* pTex
     default:                  editStyle = ES_CENTER; break;
   }
 
-  mParamEditWnd = CreateWindow("EDIT", pString, ES_AUTOHSCROLL /*only works for left aligned text*/ | WS_CHILD | WS_VISIBLE | ES_MULTILINE | editStyle,
+  WCHAR buf[MAX_PARAM_LEN];
+  MultiByteToWideChar(CP_UTF8, 0, pString, -1, buf, MAX_PARAM_LEN);
+  mParamEditWnd = CreateWindowW(L"EDIT", buf, ES_AUTOHSCROLL /*only works for left aligned text*/ | WS_CHILD | WS_VISIBLE | ES_MULTILINE | editStyle,
                                pTextRect->L, pTextRect->T, pTextRect->W()+1, pTextRect->H()+1,
                                mPlugWnd, (HMENU) PARAM_EDIT_ID, mHInstance, 0);
 
@@ -976,7 +980,7 @@ void IGraphicsWin::CreateTextEntry(IControl* pControl, IText* pText, IRECT* pTex
 
   SetFocus(mParamEditWnd);
 
-  mDefEditProc = (WNDPROC) SetWindowLongPtr(mParamEditWnd, GWLP_WNDPROC, (LONG_PTR) ParamEditProc);
+  mDefEditProc = (WNDPROC) SetWindowLongPtrW(mParamEditWnd, GWLP_WNDPROC, (LONG_PTR) ParamEditProc);
   SetWindowLongPtr(mParamEditWnd, GWLP_USERDATA, 0xdeadf00b);
 
   //DeleteObject(font);

Quote:
Originally Posted by stw View Post
You qualified for unicode enabled license of my plug if you like
I don't think you mentioned which plug-in this is for, but nevertheless: Yes, please!
Tale is online now   Reply With Quote
Old 08-13-2020, 09:01 AM   #9
stw
Human being with feelings
 
stw's Avatar
 
Join Date: Apr 2012
Posts: 279
Default

Quote:
Originally Posted by Tale View Post
I don't think you mentioned which plug-in this is for, but nevertheless: Yes, please!
https://forum.cockos.com/showpost.ph...&postcount=111

I'll PM you!
stw is offline   Reply With Quote
Old 08-13-2020, 11:13 AM   #10
Nonlinear
Human being with feelings
 
Join Date: Apr 2018
Posts: 396
Default

Quote:
Originally Posted by stw View Post
Hi,
i recently released my new plug and ran into some problems i haven't considered right before.
To unlock my plug the user has to input his name from which I calc a hash and some other stuff. I take the string from an ITextInputControl.
Xcode/mac properly translates all unicode characters into the corresponding bytes. Unfortunately VS/win does not.
I am also using a text entry box for entering license codes and have found the opposite is happening in some cases, i.e., text entries that work fine on Windows sometimes do not work properly on Mac. For example spaces in names on Windows sometimes get replaced with hyphens on Mac. I guess what works for you depends on WHERE you developed your plugin (I develop on Windows where Mac is the "the other system" but I know many/most develop on Mac where Windows is "the other system").

Will the code fix presented here by Tale produce better cross-platform consistency or can it break what currently works for me on Windows?
Nonlinear is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 12:49 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.