Braingolf, 42 40 bytes

#"#H#e#l#l#o#,# #W#o#r#l#d#!#" $_ <$_& @

Try it online!

After removing odd characters:

"Hello, World!"$ $&@

Try it online!

##Explanation Full program:

#"#H#e#l#l#o#,# #w#o#r#l#d#!#" pushes "Hello, World!" (including quotes)

$ adds the silent modifier to the next operator
space does nothing
_ Pops and prints the last item on the stack, but due to silent mode, does not print
< moves the first item on the stack to the end of the stack

This means that $_ <$_ will remove both quotes from the stack

& adds the greedy modifier to the next operator
@ pops and prints the last item on the stack as an ASCII character
  greedy modifier means it prints the entire stack.

With characters removed:

"Hello, World!" Pushes Hello, World!
Spaces are no-ops
&@ Prints entire stack as ASCII
@ is not affected by the silent modifier (apparently)

It feels really cheap using no-ops for this, but it's better than the Java/Python comment solutions imo

Braingolf, 42 40 bytes

#"#H#e#l#l#o#,# #W#o#r#l#d#!#" $_ <$_& @

Try it online!

After removing odd characters:

"Hello, World!"$ $&@

Try it online!

Explanation

Full program:

#"#H#e#l#l#o#,# #w#o#r#l#d#!#" pushes "Hello, World!" (including quotes)

$ adds the silent modifier to the next operator
space does nothing
_ Pops and prints the last item on the stack, but due to silent mode, does not print
< moves the first item on the stack to the end of the stack

This means that $_ <$_ will remove both quotes from the stack

& adds the greedy modifier to the next operator
@ pops and prints the last item on the stack as an ASCII character
  greedy modifier means it prints the entire stack.

With characters removed:

"Hello, World!" Pushes Hello, World!
Spaces are no-ops
&@ Prints entire stack as ASCII
@ is not affected by the silent modifier (apparently)

It feels really cheap using no-ops for this, but it's better than the Java/Python comment solutions imo