The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, graduate student. However, this was an improvement over previous models, whose capability was closer to an actually incompetent graduate student.
Hilarious.
I had a previous discussion with ChatGPT that went something like this:
Me: give me an implementation of AES encryption in Java
chatGPT: outputs implementation
Me: I see 3 security problems with it, can you find them?
ChatGPT: outputs three problems
Me: okay, you found one that I missed, so there are 4 problems with it.
ChatGPT: finds 4th problem
Me: okay, now give me an implementation that fixes these problems
ChatGPT: outputs new implementation
Me: okay, but why did you hardcode the keys? And why are they strings rather than byte arrays?
ChatGPT: acknowledges mistakes and outputs new implementation
Me: that fixes the most recent mistakes, but you brought back one of the original mistakes that you identified earlier.
ChatGPT: outputs an implementation that fixes most recent mistakes
I think there was one more iteration before it finally gave me an implementation that I could not quickly see a flaw in it.
Just be careful when using AI for programming. It’s not a win when the person asking the questions needs to be an expert in subject that they are trying to get basic guidance on.
Was this with the new models launched last week? I've found that o1-mini (optimized for coding tasks) does a great job of more "tricky" tasks than previous models, which indeed fall for subtle traps. I don't have a security background, but in some perf-sensitive scenarios (in Go) I've found a big difference in asking for a code snippet that account for allocations compared to the gpt-4 and claude 3.5 models.
Sorry, I didn’t say when that happened. It was a bit more than a year ago. I was hoping to blog about it some day but I never got around to it. But after reading some other examples of common mistakes AI makes, my motivation for blogging about it waned.
Try out the new models and then come back and provide your opinion, whats the point of putting up code that it made one year ago when we have got iterations that are better then those old models? Like using a 2010 Nokia and saying "see mobile phones suck"
Keep in mind I do not know anything about this, but the new model gave me this:
Here's a general outline of the steps involved:
Generate or Obtain a Secret Key:
You can generate a secret key using the KeyGenerator class.
Alternatively, derive a key from a password using a key derivation function like PBKDF2 with SecretKeyFactory.
Create a Cipher Instance:
Use the Cipher class with the transformation string "AES" or specify the mode and padding like "AES/CBC/PKCS5Padding".
Initialize the Cipher:
Initialize the cipher for encryption or decryption using cipher.init() with the appropriate mode (Cipher.ENCRYPT_MODE or Cipher.DECRYPT_MODE), the secret key, and an initialization vector (IV) if required by the mode.
Encrypt or Decrypt Data:
Use cipher.doFinal() to process the input data and obtain the encrypted or decrypted bytes.
Handle Encoding:
If necessary, encode the resulting bytes to a readable format using Base64 encoding.
Key Considerations:
Security Practices:
Always use a secure random number generator for key and IV generation (SecureRandom).
When using modes like CBC, ensure the IV is random and unique for each encryption.
Store and handle keys securely to prevent unauthorized access.
Exception Handling:
Be prepared to handle exceptions such as NoSuchAlgorithmException, NoSuchPaddingException, and InvalidKeyException.
Certainly! Below is a simple example of how to implement AES encryption and decryption in Java using the javax.crypto library. This example uses AES in CBC mode with PKCS5 padding.
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.IvParameterSpec;
import java.security.SecureRandom;
import java.util.Base64;
public class AESExample {
public static void main(String[] args) {
try {
// Generate a new AES key
SecretKey secretKey = generateKey(256);
// Generate a new IV (Initialization Vector)
IvParameterSpec ivParameterSpec = generateIv();
// The plaintext to be encrypted
String plaintext = "Hello, World!";
// Encrypt the plaintext
String ciphertext = encrypt(plaintext, secretKey, ivParameterSpec);
System.out.println("Encrypted Text: " + ciphertext);
// Decrypt the ciphertext
String decryptedText = decrypt(ciphertext, secretKey, ivParameterSpec);
System.out.println("Decrypted Text: " + decryptedText);
} catch (Exception e) {
e.printStackTrace();
}
}
// Method to generate a SecretKey
public static SecretKey generateKey(int n) throws Exception {
KeyGenerator keyGenerator = KeyGenerator.getInstance("AES");
keyGenerator.init(n); // Key size: 128, 192, or 256 bits
return keyGenerator.generateKey();
}
// Method to generate an Initialization Vector (IV)
public static IvParameterSpec generateIv() {
byte[] iv = new byte[16]; // AES block size is 16 bytes
new SecureRandom().nextBytes(iv);
return new IvParameterSpec(iv);
}
// Method to perform AES encryption
public static String encrypt(String plaintext, SecretKey key, IvParameterSpec iv)
throws Exception {
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, key, iv);
byte[] ciphertextBytes = cipher.doFinal(plaintext.getBytes("UTF-8"));
return Base64.getEncoder().encodeToString(ciphertextBytes);
}
// Method to perform AES decryption
public static String decrypt(String ciphertext, SecretKey key, IvParameterSpec iv)
throws Exception {
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, key, iv);
byte[] plaintextBytes = cipher.doFinal(Base64.getDecoder().decode(ciphertext));
return new String(plaintextBytes, "UTF-8");
}
}
Explanation:
Key Generation:
The generateKey method creates a new AES SecretKey with the specified key size (128, 192, or 256 bits).
Ensure your Java environment supports the desired key size. For key sizes above 128 bits, you may need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files.
Initialization Vector (IV):
The generateIv method creates a random 16-byte IV, which is required for AES in CBC mode.
The IV does not need to be secret but should be unpredictable and unique for each encryption operation.
Encryption:
The encrypt method initializes the Cipher in ENCRYPT_MODE with the given key and IV.
It encrypts the plaintext and encodes the resulting byte array to a Base64 string for easy representation.
Decryption:
The decrypt method initializes the Cipher in DECRYPT_MODE with the same key and IV used for encryption.
It decodes the Base64 ciphertext and decrypts it to retrieve the original plaintext.
Encoding:
Base64 encoding is used to convert binary data into a string format that can be easily stored or transmitted.
Important Notes:
Exception Handling:
In a production environment, handle exceptions appropriately rather than using printStackTrace().
Security Considerations:
Key Management: Securely store and handle the secret key and IV. Do not hard-code them in your source code or expose them publicly.
IV Transmission: The IV should be sent along with the ciphertext to the recipient for decryption. It can be prefixed to the ciphertext or transmitted separately.
Randomness: Use a cryptographically secure random number generator (SecureRandom) for generating keys and IVs.
Character Encoding:
Ensure consistent character encoding (e.g., UTF-8) when converting between strings and byte arrays.
Dependencies:
Make sure to import the necessary classes from javax.crypto, java.security, and java.util packages.
PKCS5Padding is commonly used, but ensure it matches on both encryption and decryption sides.
Be cautious with modes and padding schemes to prevent vulnerabilities.
Algorithm Modes:
CBC (Cipher Block Chaining) mode is used in this example. Other modes like GCM (Galois/Counter Mode) provide built-in authentication and are recommended for modern applications.
Authenticated Encryption:
Consider using AES in GCM mode ("AES/GCM/NoPadding") for authenticated encryption, which provides data integrity in addition to confidentiality.
Example with AES GCM Mode:
// For AES GCM Mode
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
// GCMParameterSpec spec = new GCMParameterSpec(128, iv);
// cipher.init(Cipher.ENCRYPT_MODE, key, spec);
It’s pretty good, all except for one little issue that it acknowledges at the end:
CBC (Cipher Block Chaining) mode is used in this example. Other modes like GCM (Galois/Counter Mode) provide built-in authentication and are recommended for modern applications.
Why would it give us CBC if it knows that GCM is better?
coming from a place where i have no knowledge of this specifically, i would say it has supplied enough information that I can then follow up on my own.
Also, I only asked for an example. with careful prompting, you could probably get a decent implementation.
Ah, that likely explains the common errors. Even gpt-4 then (paid only) regularly gaffed with non-boilerplate code. I’d say it’s changed substantially now, but like any tool, effective use requires some practice (like how to prompt which model family), and many folks — juniors in particular — risk emitting subtly wrong code at a large scale because of that.
Next time ask it in the original promo to review its own work, think out loud, etc. I’m being brief here but this could be a long description. It won’t fix the issue fully but it can often make significant improvements
Still, this is quite a capable human being. A mediocre graduate can do a metric ton of stuff in almost any industry.
One thing that that makes the difference is that it does it in seconds, so it doesn't really compare.
I've been using Cloud and ChatGPT as junior devs, and they are absolutely awesome at it whit this expectation. The difference is that if I give a task to a real junior dev I'll have to wait a day to see the results.
ScottContini@reddit
Hilarious.
I had a previous discussion with ChatGPT that went something like this:
Me: give me an implementation of AES encryption in Java
chatGPT: outputs implementation
Me: I see 3 security problems with it, can you find them?
ChatGPT: outputs three problems
Me: okay, you found one that I missed, so there are 4 problems with it.
ChatGPT: finds 4th problem
Me: okay, now give me an implementation that fixes these problems
ChatGPT: outputs new implementation
Me: okay, but why did you hardcode the keys? And why are they strings rather than byte arrays?
ChatGPT: acknowledges mistakes and outputs new implementation
Me: that fixes the most recent mistakes, but you brought back one of the original mistakes that you identified earlier.
ChatGPT: outputs an implementation that fixes most recent mistakes
I think there was one more iteration before it finally gave me an implementation that I could not quickly see a flaw in it.
Just be careful when using AI for programming. It’s not a win when the person asking the questions needs to be an expert in subject that they are trying to get basic guidance on.
phillipcarter2@reddit
Was this with the new models launched last week? I've found that o1-mini (optimized for coding tasks) does a great job of more "tricky" tasks than previous models, which indeed fall for subtle traps. I don't have a security background, but in some perf-sensitive scenarios (in Go) I've found a big difference in asking for a code snippet that account for allocations compared to the gpt-4 and claude 3.5 models.
ScottContini@reddit
Sorry, I didn’t say when that happened. It was a bit more than a year ago. I was hoping to blog about it some day but I never got around to it. But after reading some other examples of common mistakes AI makes, my motivation for blogging about it waned.
Aggressive_Soil_5134@reddit
Try out the new models and then come back and provide your opinion, whats the point of putting up code that it made one year ago when we have got iterations that are better then those old models? Like using a 2010 Nokia and saying "see mobile phones suck"
SPQR-VVV@reddit
Keep in mind I do not know anything about this, but the new model gave me this:
Here's a general outline of the steps involved:
KeyGenerator
class.Alternatively, derive a key from a password using a key derivation function like PBKDF2 with
SecretKeyFactory
.Create a Cipher Instance:
Use the
Cipher
class with the transformation string"AES"
or specify the mode and padding like"AES/CBC/PKCS5Padding"
.Initialize the Cipher:
Initialize the cipher for encryption or decryption using
cipher.init()
with the appropriate mode (Cipher.ENCRYPT_MODE
orCipher.DECRYPT_MODE
), the secret key, and an initialization vector (IV) if required by the mode.Encrypt or Decrypt Data:
Use
cipher.doFinal()
to process the input data and obtain the encrypted or decrypted bytes.Handle Encoding:
Key Considerations:
SecureRandom
).Store and handle keys securely to prevent unauthorized access.
Exception Handling:
NoSuchAlgorithmException
,NoSuchPaddingException
, andInvalidKeyException
.References for Further Reading:
When I asked for a sample I got this:
Certainly! Below is a simple example of how to implement AES encryption and decryption in Java using the
javax.crypto
library. This example uses AES in CBC mode with PKCS5 padding.Explanation:
generateKey
method creates a new AESSecretKey
with the specified key size (128, 192, or 256 bits).Ensure your Java environment supports the desired key size. For key sizes above 128 bits, you may need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files.
Initialization Vector (IV):
generateIv
method creates a random 16-byte IV, which is required for AES in CBC mode.The IV does not need to be secret but should be unpredictable and unique for each encryption operation.
Encryption:
encrypt
method initializes theCipher
inENCRYPT_MODE
with the given key and IV.It encrypts the plaintext and encodes the resulting byte array to a Base64 string for easy representation.
Decryption:
decrypt
method initializes theCipher
inDECRYPT_MODE
with the same key and IV used for encryption.It decodes the Base64 ciphertext and decrypts it to retrieve the original plaintext.
Encoding:
Important Notes:
In a production environment, handle exceptions appropriately rather than using
printStackTrace()
.Security Considerations:
Randomness: Use a cryptographically secure random number generator (
SecureRandom
) for generating keys and IVs.Character Encoding:
Dependencies:
javax.crypto
,java.security
, andjava.util
packages.Sample Output:
Additional Tips:
Be cautious with modes and padding schemes to prevent vulnerabilities.
Algorithm Modes:
CBC (Cipher Block Chaining) mode is used in this example. Other modes like GCM (Galois/Counter Mode) provide built-in authentication and are recommended for modern applications.
Authenticated Encryption:
"AES/GCM/NoPadding"
) for authenticated encryption, which provides data integrity in addition to confidentiality.Example with AES GCM Mode:
You tell me if it is better now than before.
ScottContini@reddit
It’s pretty good, all except for one little issue that it acknowledges at the end:
Why would it give us CBC if it knows that GCM is better?
SPQR-VVV@reddit
coming from a place where i have no knowledge of this specifically, i would say it has supplied enough information that I can then follow up on my own.
Also, I only asked for an example. with careful prompting, you could probably get a decent implementation.
phillipcarter2@reddit
Ah, that likely explains the common errors. Even gpt-4 then (paid only) regularly gaffed with non-boilerplate code. I’d say it’s changed substantially now, but like any tool, effective use requires some practice (like how to prompt which model family), and many folks — juniors in particular — risk emitting subtly wrong code at a large scale because of that.
OneNoteToRead@reddit
How’s that not a win? You were the expert and you massively reduced your workload.
nutyourself@reddit
Next time ask it in the original promo to review its own work, think out loud, etc. I’m being brief here but this could be a long description. It won’t fix the issue fully but it can often make significant improvements
GeorgiLubomirov@reddit
Still, this is quite a capable human being. A mediocre graduate can do a metric ton of stuff in almost any industry.
One thing that that makes the difference is that it does it in seconds, so it doesn't really compare.
I've been using Cloud and ChatGPT as junior devs, and they are absolutely awesome at it whit this expectation. The difference is that if I give a task to a real junior dev I'll have to wait a day to see the results.
ycatbin_k0t@reddit
GPT-4o2 will solve the world code hunger, I swear, bro. Just one more Nvidia share, and you won't have a job for the next 4 years