For scholarly papers, a fair number of us (yours truly included) still swear by the cross-platform LaTeX typesetting system rather than by Microzift Word or any of its competitors. (Often the feature we LaTeX-users love most about it is the one non-LaTeX users hate most: strict separation between content and formatting. LaTeX is basically a markup language like HTML, but with a kajillion more possibilities, especially in terms of mathematical typesetting and cross-referencing. A not-so-short-introduction can be found
here.)
A comprehensive directory of LaTeX-on-the-Macintosh resources can be found
here. The most popular Classic Mac OS implementation of LaTeX, TeXtures, is no longer being developed for Mac OS X: however, Mac OS X being basically a version of Unix, the most popular Unix implementation teTeX (which comes preinstalled on most Linux distributions) has been ported to Mac OS X virtually as soon as the latter came out. teTeX is open source.
The most convenient way to install it on a Mac OS X machine is probably Gerben Wierda's
i-Installer for gwTeX. This is an interactive installer that will download all necessary pieces and install them. (The installations involve "compiling" the most common TeX formats.) If you choose this option, you should at minimum select the "TeX" and "GhostScript 8" packages. If you are planning to use figures in other formats than PostScript or Encapsulated PostScript, also grab "Freetype2", "libvmv and iconv conversion support", and "ImageMagick". You may also want to grab the "CM Super Fonts" package, which are fully scalable PostScript Type 1 fonts for the Computer Modern family (TeX and LaTeX's signature font).
Note that the "TeX" installation will take a nontrivial amount of time (5 to 15 minutes, depending on the speed of your CPU and hard disk). If you are concerned that the installation is stuck (in truth, it's doing plenty), turn on "Geek mode" and click on the "Subprocess output" tab. You'll see plenty of activity (probably gobbledygook if you're a (La)TeX novice).
Another alternative is to grab the "fat installer", a 60 MB download which already has the TeX and Ghostscript 8 packages bundled. A HTTP mirror is available
here at Yale.
If you are an experienced TeX/LaTeX user and have accumulated a number of personalized style files/custom fonts/..., make a subdirectory ~/Library/texmf/ if it does not exist yet and install them in appropriate subdirectories there.
Now what you still need (unless you're even more of a Unix geek than I am :-)) is a graphical front-end. There are a number of alternatives: I personally use the open-source
TeXShop (although some may prefer its spinoff
iTeXMac).
Note that the default typesetting mode of these applications is pdflatex (since Mac OS X has native PDF support built into its windowing system). This is great for documents that do not involve figures (just text and math), but if you routinely include figures in your papers then you may want to change the default typesetting mode to "TeX+Ghostscript" in "Preferences".