[C#/asm] Trying to Translate an Algorithm

Posted by urmuther112@reddit | learnprogramming | View on Reddit | 19 comments

I am trying to translate this binary insertion sort written with C# to assembly (MASM if that matters). Here is what I have so far. To be quite honest this spaghetti'd together using my basic knowledge and all the hard to read solutions I could find on Google. My issue lies in not knowing how to print the sorted array to the console. When I went to search up how others have done it, the solutions would be very different from each other making it hard to implement in my own code. I then tried going to GPT with this issue but it kept telling to use syscalls and VGA memory that I know nothing about. Is there not just a way to convert the array into ASCII and push/pop them to the console? I just wanna see if the sort actually works.

[-]

Careless_Quail_4830@reddit

You can get this done the way you asked it in several ways, such as directly calling windows functions (eg WriteConsoleA) or linking with the C runtime so you can use printf.

What I normally do, as long as a project doesn't need to be purely in assembly, is write a C++ program that calls the assembly code. All the things that are annoying to do in assembly stay on the C++ side, and calling a function that is implemented in assembly from C++ is easy. This makes it easier to print results, generate test data, run tests, and so on.

I then tried going to GPT with this issue but it kept telling to use syscalls and VGA memory that I know nothing about.

It's getting confused anyway.

[-]

urmuther112@reddit (OP)

I went with the Windows functions but I am now having issues getting the kernel32 files to open. I have already added the .lib to the linker and the .inc pathway is included in the masm settings. I also have WriteConsoleA, ExitProcess, and GetStdHandle all declared directly so from my understanding the .inc shouldn't matter. But either way I've got the include statements at the top for both files for redundancy until I figure this out.

[-]

Careless_Quail_4830@reddit

If your code is still similar to what you added a link to, it's because you have the wrong prototype for WriteConsoleA. That would always be kind of bad but matters even more in 32-bit code with the stdcall calling convention because it affects the link name of the symbol: you end up trying to link with WriteConsoleA@12 and that doesn't exist.

If I change the prototype to this, I get it to link:

WriteConsoleA PROTO, hConsoleOutput: DWORD, lpBuffer: PTR BYTE, nNumberOfCharsToWrite: DWORD, lpWritten: PTR DWORD, reserved: PTR BYTE

(just pass zeroes for the last two parameters when you invoke it)

The code crashes with an access violation but hey it's progress.

[-]

urmuther112@reddit (OP)

I got rid of the first access violation but now I got another in the write_loop that I'm not sure how to handle. It is happening on this line: mov eax, [array + esi * 4]

I suspect it has something to do with going out of bounds on the array.

[-]

Careless_Quail_4830@reddit

One issue with the printing code is that calls to stdcall functions (including WriteConsoleA) can overwrite the contents of eax, ecx, and edx. That loop has (in the version that I see anyway, perhaps you've changed it by now) keeps the length of the array in ecx, but that's getting rekt after printing and the loop keeps looping too long.

Another thing is that number_to_ascii doesn't write the string at the start of the buffer. That's fine, but the call to WriteConsoleA to print that string needs to take that into account.

Also you need an INVOKE GetStdHandle, -11 somewhere, -11 does not go directly into WriteConsoleA etc that expect a handle

[-]

urmuther112@reddit (OP)

So I could fix the issue by resetting ecx to the lengthof array at the beginning of the write_loop instead of outside of it? And I also made the change to INVOKE GetStdHandle.

[-]

Careless_Quail_4830@reddit

That would do it, cmp esi, LENGTHOF array also avoids that issue (assuming esi isn't getting nuked anyway)

[-]

urmuther112@reddit (OP)

Made the changes and it doesn't throw any errors however the array still doesn't print and I'm not sure what a garbage value looks like but esi = 00AE5038 kinda makes me think its getting changed by WriteConsoleA. Is there a way to hold on to the correct value before it gets nuked in order to restore it to the right value on the next iteration?

[-]

Careless_Quail_4830@reddit

Now mov esi, ecx is overwriting esi right before the comparison

[-]

urmuther112@reddit (OP)

The idea was to load ecx with 0 outside the loop and then move the 0 into esi once inside. After esi does it thing, ecx would hold on to the value so that it could restore esi on the next go around. Thinking about it now ecx is also probably getting dumped on by WriteConsole so I'm back to square one.

[-]

Careless_Quail_4830@reddit

Aside from trying to solve the puzzle so that everything is in registers, if that's what you're doing, you can also use some other storage locations. That may make things easier, less "juggling values" and more just using them.

You can push things before a call and pop them afterwards to prevent them getting overwritten.

You can create some LOCAL variables, like this:

testproc PROC
    LOCAL variable1: DWORD, variable2: DWORD

    mov variable1, 42
    mov ecx, variable1
    mov variable2, ecx  ; the local variables are in memory, so you cannot directly mov from one into the other
            imul eax, variable2, 3

    ret
testproc ENDP

If you do that, MASM sets up the stack frame for you and it adds leave before the ret (even if you have multiple returns, they all get a leave).

[-]

urmuther112@reddit (OP)

I didn't even think about push/popping. I made the changes and it fixed the error but still nothing prints. This lead me to do this in the main PROC:

main PROC
    INVOKE GetStdHandle, -11
    mov ebx, eax
    mov eax, 12345
    call number_to_ascii
    INVOKE WriteConsoleA, ebx, OFFSET buffer, 5, 0, 0

And I still can't get anything to print. I guess I'm gonna start going over number_to_ascii since that seems to be the next issue.

[-]

Careless_Quail_4830@reddit

Did you change number_to_ascii to put the string at the start of the buffer? You don't have to do it that way, but you're printing from the start of the buffer now, so if the string doesn't go there then the print picks up the wrong data.

I changed the lpBuffer argument of the print from OFFSET buffer to edi, then the program printed Sorted array:%d 0 12 17 23 31 37 72

[-]

urmuther112@reddit (OP)

I was getting somewhere by printing garbage values but at least it was the correct amount of values. This comment was the final piece of the puzzle and I can finally start on the 4-6 pages of documentation and lessons learned from this project. Thank you for the quick responses since this is due today lmao.

[-]

randomjapaneselearn@reddit

to use syscalls and VGA memory that I know nothing about

it depends if you are in dos, linux or windows and also if it's 32 bit or 64bit.

if you are on windows for example you can use microsoft provided API to print https://learn.microsoft.com/en-us/windows/console/writeconsole (see the other user answer), you still need to convert numbers to ascii.

don't forget that you can check everything even without printing by using a debugger like https://x64dbg.com/, you execute your code step by step and you can see if it works or where are mistakes if any.

if you are on linux you can use syscalls (see here https://hackeradam.com/x86-64-linux-syscalls/ or other similar links)

[-]

urmuther112@reddit (OP)

I went the Windows API route but now I am getting an error for an unresolved external. I have already added kernel32 to the linker and rebuilt the project. I have also changed all my WriteConsoleA calls to use INVOKE instead to see if that helped but I am still getting the same error.

[-]

randomjapaneselearn@reddit

did you add both?

include kernel32.inc
includelib kernel32.lib

[-]

urmuther112@reddit (OP)

I only did the .lib in the project properties. Am I supposed to add these lines to my code aswell?

[-]

Lumpy_Ad7002@reddit

Printing is one of those things that's hard in assembler. Fortunately, yours is also a common question, and StackOverflow has the answers

https://stackoverflow.com/questions/74984639/how-to-call-windows-api-in-x64-masm-program